Difference between revisions of "Introduction and notation"

Latest revision as of 13:54, 7 June 2013

(If you are experiencing problems with the display of the mathematical formula, you can either try to use another browser, or use this link which should work smoothly: http://popix.lixoft.net)

Different representations of the same model

The description of a model requires variables such as observations $(y_i)$, individual parameters $(\psi_i)$, population parameters $\theta$, covariates $(c_i)$, etc.

Tasks to be performed (estimation, simulation, likelihood calculation, etc.) involve these variables. Algorithms used to perform these tasks can use different parameterizations, i.e., different mathematical representations of the same model. We will see that depending on the task, some mathematical representations are more suitable than others.

There exists for a modeler a natural parametrization involving a vector of individual parameters $\psi_i$ which have a physical or biological meaning (rate, volume, bioavailability, etc.). We will denote by $\psi$-representation the mathematical representation of the model which uses $\psi_i$:

$ \pyipsii(y_i , \psi_i ; \theta) = \pcyipsii(y_i | \psi_i)\ppsii( \psi_i ; \theta, c_i). $

(1)

When there exists a transformation $h: \Rset^d \to \Rset^d$ such that $\phi_i=h(\psi_i)$ is a Gaussian vector, we can use equivalently the $\phi$-representation which involves the transformed parameters (log-rate, log-volume, logit-bioavailability, etc.) and now represents the joint distribution of $y_i$ and $\phi_i$:

$ \pyiphii(y_i , \phi_i ; \theta, c_i) = \pcyiphii(y_i | \phi_i)\pphii( \phi_i ; \theta, c_i), $

(2)

where $ \phi_i =h(\psi_i) \sim {\cal N}( \mu(\beta,c_i) , \Omega)$ and $\theta=(\beta,\Omega)$.

There is yet another mathematical representation which uses the vector of random effects $\eta_i$ to represent the individual parameters model:

$\begin{eqnarray} \phi_i &=& \mu(\beta,c_i) + \eta_i , \end{eqnarray}$

where $\eta_i \sim {\cal N}( 0 , \Omega)$. This $\eta$-representation leads to the joint distribution of $y_i$ and $\eta_i$:

$ \pyietai(y_i , \eta_i ; \theta, c_i) = \pcyietai(y_i | \eta_i;\beta,c_i)\petai( \eta_i ; \Omega). $

(3)

We can see that the fixed effects $\beta$ now appear in the conditional distribution of the observations. This will have a strong impact on tasks such as estimation of population parameters since a sufficient statistic for estimating $\beta$ derived from this representation will be a function of the observations $\by$, as opposed to the other representations, where the sufficient statistic is a function of the individual parameters $\bpsi$ (or equivalently, $\bphi$).

In the $\psi$-representation (1), if the model $\ppsii( \psi_i ; \theta, c_i)$ is not a regular statistical model (some components of $\psi_i$ may have no variability, or more generally $\Omega$ may not be positive definite), no sufficient statistic $S(\psi_i)$ for estimating $\theta$ exists. Thus, estimation algorithms will not use representation (1) in these cases, but another decomposition into regular statistical models.

Some examples

Consider the following model for continuous data with a constant error model:

$\begin{eqnarray} y_{ij} &\sim& {\cal N}(f(t_{ij},\phi_i) ,a_i^2) \\ \phi_i &\sim& {\cal N}(\beta, \Omega) \\ a_i &\sim& p_a(\, \cdot \, ; \theta_a) . \end{eqnarray}$

Here, the variance of the residual error is a random variable. The vector of individual parameters is $(\phi_i, a_i)$ and the vector of population parameters is $\theta=(\beta,\Omega,\theta_a)$. Assuming that $\Omega$ is positive definite, the joint model of $y_i$, $\phi_i$ and $a_i$ can be decomposed as a product of three regular models:

$ \pyiphii(y_i , \phi_i, a_i ; \theta) = \pcyiphii(y_i | \phi_i ,a_i)\pphii( \phi_i ; \beta, \Omega)\pmacro(a_i ; \theta_a). $

Assume instead that the variance of the residual error is fixed for the whole population:

$\begin{eqnarray} y_{ij} &\sim& {\cal N}(f(t_{ij},\phi_i) ,a^2) . \end{eqnarray}$

The vector of population parameters is now $\theta=(\beta,\Omega,a)$ and the joint model of $y_i$ and $\phi_i$ can be decomposed as

$ \pyiphii(y_i , \phi_i ; \theta) = \pcyiphii(y_i | \phi_i ; a)\pphii( \phi_i ; \beta, \Omega). $

Suppose that some components of $\phi_i$ have no inter-individual variability. More precisely, let $\phi_i=(\phi_i^{(1)} \phi_i^{(0)})$ and $\beta=(\beta_1,\beta_0)$, such that

$\begin{eqnarray} \phi_i^{(1)} &\sim& {\cal N}(\beta_1, \Omega_1) \\ \phi_i^{(0)} &=& \beta_0 , \end{eqnarray}$

and $\Omega_1$ is positive definite. Here, $\theta=(\beta_1,\beta_0,\Omega_1,a)$ and

$ \pyiphii(y_i , \phi_i^{(1)} ; \theta) = \pcyiphii(y_i | \phi_i^{(1)} ; \beta_0, a)\pphii( \phi_i^{(1)} ; \beta_1, \Omega_1). $

Assume instead that $\phi_i = (\phi_{i,1}, \phi_{i,2})$, where

$\begin{eqnarray} \phi_{i,1} &=& \beta_1 + \omega_1\eta_i \\ \phi_{i,2} &=& \beta_2 + \omega_2\eta_i , \end{eqnarray}$

and $\eta_i \sim {\cal N}(0,1)$. Here, the useful model is the joint distribution of $y_i$ and $\eta_i$. We can use for instance the following $\eta$-representation:

$ \pyietai(y_i , \eta_i ; \theta) = \pcyietai(y_i | \eta_i ;\theta)\petai( \eta_i), $

where $\theta= (\beta_1,\beta_2, \omega_1,\omega_2,a)$.

Some notation

We assume that the set of population parameters $\theta$ takes its values in $\Theta$, an open subset of $\Rset^m$.

Let $f : \Theta \to \Rset$ be a twice differentiable function of $\theta$. We will denote $\Dt{f(\theta)} = (\partial f(\theta)/\partial \theta_j, 1 \leq j \leq m) $ the gradient of $f$ (i.e., the vector of partial derivatives of $f$) and $\DDt{f(\theta)} = (\partial^2 f(\theta)/\partial \theta_j\partial \theta_k, 1 \leq j,k \leq m) $ the Hessian of $f$ (i.e., the square matrix of second-order partial derivatives of $f$).

The SAEM algorithm for estimating population parameters

@@ Line 1: / Line 1: @@
-<div style="font-size:12pt; font-family:garamond">
+<div style="color: #2E5894; padding-left: 1.4em; padding-right:2.2em; padding-bottom:0.8em; padding:top:1">[[Image:attention4.jpg|45px|left|link=]]
-<!-- some LaTeX macros we want to use: -->
+(If you are experiencing problems with the display of the mathematical formula, you can either try to use another browser, or use this link which should work smoothly:   http://popix.lixoft.net)
-$
+</div>
+== Different representations of the same model ==
+The description of a model requires variables such as observations $(y_i)$, individual parameters $(\psi_i)$, population parameters $\theta$, covariates $(c_i)$, etc.
+Tasks to be performed (estimation, simulation, likelihood calculation, etc.) involve these variables.
+Algorithms used to perform these tasks can use different parameterizations, i.e.,  different mathematical representations of the same model. We will see that depending on the task, some mathematical representations are more suitable than others.
-\newcommand{\prob}[1]{ \mathbb{P}\left(#1\right)}
+There exists for a modeler a natural parametrization  involving a vector of individual parameters $\psi_i$ which have a physical or biological meaning (rate, volume, bioavailability, etc.). We will denote by ''$\psi$-representation'' the mathematical representation of the model which uses $\psi_i$:
-\newcommand{\probs}[2]{ \mathbb{P}_{#1}\left(#2\right)}
-\newcommand{\esp}[1]{\mathbb{E}\left(#1\right)}
-\newcommand{\esps}[2]{\mathbb{E}_{#1}\left(#2\right)}
-\newcommand{\var}[1]{\mbox{Var}\left(#1\right)}
-\newcommand{\vars}[2]{\mbox{Var}_{#1}\left(#2\right)}
-\newcommand{\std}[1]{\mbox{sd}\left(#1\right)}
-\newcommand{\stds}[2]{\mbox{sd}_{#1}\left(#2\right)}
-\newcommand{\corr}[1]{\mbox{Corr}\left(#1\right)}
-\newcommand{\Rset}{\mbox{$\mathbb{R}$}}
-\newcommand{\Yr}{\mbox{$\mathcal{Y}$}}
-\newcommand{\teps}{\varepsilon}
-\newcommand{\like}{\cal L}
-\newcommand{\logit}{\rm logit}
-\newcommand{\transy}{u}
-\newcommand{\repy}{y^{(r)}}
-\newcommand{\brepy}{\boldsymbol{y}^{(r)}}
-\newcommand{\vari}[3]{#1_{#2}^{{#3}}}
-\newcommand{\dA}[2]{\dot{#1}_{#2}(t)}
-\newcommand{\nitc}{N}
-\newcommand{\itc}{I}
-\newcommand{\vl}{V}
-\newcommand{tstart}{t_{start}}
-\newcommand{tstop}{t_{stop}}
-\newcommand{\one}{\mathbb{1}}
-\newcommand{\hazard}{h}
-\newcommand{\cumhaz}{H}
-\newcommand{\std}[1]{\mbox{sd}\left(#1\right)}
-\newcommand{\eqdef}{\mathop{=}\limits^{\mathrm{def}}}
-\def\mlxtran{\text{MLXtran}}
-\def\monolix{\text{Monolix}}
-$
-== Different representations of the same model ==
-A description of the model involves several kinds of variables, including observations, individual parameters, population parameters, covariates,...
-The tasks to execute concern these variables (estimation of the individual parameters, of the population parameters, ...).
-The algorithms used to perform these tasks may use different parametrizations, that is, different mathematical representations of the same model. We will see that, according to the task (estimation, simulation, likelihood calculation,...) some mathematical representations are more suitable than others.
-There exist for the modeler a natural parametrization which involves a vector of individual parameters $\psi_i$ which have a physical or biological meaning (rate, volume, bioavailability,...). We will call ''$\psi$-representation'' the mathematical representation of the model which uses $\psi_i$.
 {{EquationWithRef
 |equation=<div id="eq:representation1"><math>
-\pyipsii(y_i , \psi_i ; \theta) = \pcyipsii(y_i {{!}} \psi_i)\ppsii( \psi_i ; \theta).
+\pyipsii(y_i , \psi_i ; \theta) = \pcyipsii(y_i {{!}} \psi_i)\ppsii( \psi_i ; \theta, c_i).
 </math></div>
 |reference=(1) }}
-When there exists a transformation $h: \Rset^d \to \Rset^d$ such that $\phi_i=h(\psi_i)$ is a Gaussian vector, we can use equivalently the ''$\phi$-representation''  which involves the transformed parameters (log-rate, log-volume, logit-bioavailability,...) and which represents the joint distribution of $y_i$ and $\phi_i$:
+When there exists a transformation $h: \Rset^d \to \Rset^d$ such that $\phi_i=h(\psi_i)$ is a Gaussian vector, we can use equivalently the ''$\phi$-representation''  which involves the transformed parameters (log-rate, log-volume, logit-bioavailability, etc.) and now represents the joint distribution of $y_i$ and $\phi_i$:
 {{EquationWithRef
 |equation=<div id="eq:representation2"><math>
-\pyiphii(y_i , \phi_i ; \theta, c_i) = \pcyiphii(y_i {{!}} \phi_i)\pphii( \phi_i ; \theta, c_i).
+\pyiphii(y_i , \phi_i ; \theta, c_i) = \pcyiphii(y_i {{!}} \phi_i)\pphii( \phi_i ; \theta, c_i),
 </math></div>
 |reference=(2) }}
 where $ \phi_i =h(\psi_i) \sim {\cal N}( \mu(\beta,c_i) , \Omega)$ and $\theta=(\beta,\Omega)$.
-Another mathematical representation uses the vector of random effects $\eta_i$ to represent the model of the individual parameters:
+There is yet another mathematical representation which uses the vector of random effects $\eta_i$ to represent the individual parameters model:
 {{Equation1
 |equation=<math>\begin{eqnarray}
-\phi_i &=& \mu(\beta,c_i) + \eta_i
+\phi_i &=& \mu(\beta,c_i) + \eta_i ,
 \end{eqnarray}</math> }}
-where $\eta_i \sim {\cal N}( 0 , \Omega)$.
+where $\eta_i \sim {\cal N}( 0 , \Omega)$. This ''$\eta$-representation'' leads to the joint distribution of $y_i$ and $\eta_i$:
-The ''$\eta$-representation'' then represents the joint distribution of $y_i$ and $\eta_i$:
 {{EquationWithRef
 |equation=<div id="eq:representation3"><math>
 \pyietai(y_i , \eta_i ; \theta, c_i) = \pcyietai(y_i {{!}} \eta_i;\beta,c_i)\petai( \eta_i ; \Omega).
 </math></div>
-|reference=(3)}}
+|reference=(3) }}
-One can see that the fixed effects $\beta$ now appear in the conditional distribution of the observations.
+We can see that the fixed effects $\beta$ now appear in the conditional distribution of the observations. This will have a strong impact on tasks such as estimation of  population parameters since a sufficient statistic for estimating $\beta$ derived from this representation  will be a function of the observations $\by$, as opposed to the other representations, where the sufficient statistic is a function of the individual parameters $\bpsi$ (or equivalently, $\bphi$).
-This will have a strong impact for tasks such as estimation of the population parameters since a sufficient statistic derived from this representation for estimating $\beta$ will be a function of the observations $\by$, contrary to the other representations for which the sufficient statistic is a function of the individual parameters $\bpsi$ (or equivalently $\bphi$).
-In the $\psi$-representation [[#eq:representation1|(1)]], if model $\ppsii( \psi_i ; \theta, c_i)$ is not a regular statistical model (some components of $\psi_i$ may have no variability for instance, or more generally $\Omega$ may not be positive definite),  there does not exist any sufficient statistic $S(\psi_i)$ for estimating $\theta$. Thus, algorithms for estimation will not use the representation as in [[#eq:representation1|(1)]], but another decomposition into regular statistical models.
+In the $\psi$-representation [[#eq:representation1|(1)]], if the model $\ppsii( \psi_i ; \theta, c_i)$ is not a regular statistical model (some components of $\psi_i$ may have no variability, or more generally $\Omega$ may not be positive definite),  no sufficient statistic $S(\psi_i)$ for estimating $\theta$ exists. Thus, estimation algorithms  will not use  representation [[#eq:representation1|(1)]] in these cases, but another decomposition into regular statistical models.
-<!-- %L'art du modeler est de représenter mathématiquement le modèle qu'il souhaite décrire. L'art du methods and software developer sera d'identifier la représentation optimale de ce modèle -->
 {{Example
 |title=Some examples
 |text=
+<ol>
-. Consider the following model for continuous data with a constant error model:
+<li> Consider the following model for continuous data with a constant error model: </li>
 {{Equation1
@@ Line 99: / Line 59: @@
 y_{ij} &\sim& {\cal N}(f(t_{ij},\phi_i) ,a_i^2) \\
 \phi_i &\sim& {\cal N}(\beta, \Omega) \\
-a_i &\sim& p_a(\, \cdot \, ; \theta_a)
+a_i &\sim& p_a(\, \cdot \, ; \theta_a) .
 \end{eqnarray}</math> }}
-: Here, the variance of the residual error is a random variable. The vector of individual parameters is $(\phi_i, a_i)$ and the vector of population parameters is $\theta=(\beta,\Omega,\theta_a)$. Assuming that $\Omega$ is positive definite, the joint model of $y_i$ and $(\psi_i , a_i)$ can be decomposed as a product of three regular models:
+Here, the variance of the residual error is a random variable. The vector of individual parameters is $(\phi_i, a_i)$ and the vector of population parameters is $\theta=(\beta,\Omega,\theta_a)$. Assuming that $\Omega$ is positive definite, the joint model of $y_i$, $\phi_i$ and $a_i$ can be decomposed as a product of three regular models:
 {{Equation1
 |equation=<math>
-\pyiphii(y_i , \phi_i, a_i ; \theta) = \pcyiphii(y_i {{!}} \phi_i ,a_i)\pphii( \phi_i ; \beta, \Omega)p_a(a_i ; \theta_a).
+\pyiphii(y_i , \phi_i, a_i ; \theta) = \pcyiphii(y_i {{!}} \phi_i ,a_i)\pphii( \phi_i ; \beta, \Omega)\pmacro(a_i ; \theta_a).
 </math> }}
+<br>
-.  Assume now that the variance  of the residual error is fixed in the population
+<li>Assume instead that the variance  of the residual error is fixed for the whole population:</li>
 {{Equation1
 |equation=<math>\begin{eqnarray}
-y_{ij} &\sim& {\cal N}(f(t_{ij},\phi_i) ,a^2)
+y_{ij} &\sim& {\cal N}(f(t_{ij},\phi_i) ,a^2) .
 \end{eqnarray}</math> }}
-:The vector of population parameters is $\theta=(\beta,\Omega,a)$ and the joint model of $y_i$ and $\phi_i$ can be decomposed as
+The vector of population parameters is now $\theta=(\beta,\Omega,a)$ and the joint model of $y_i$ and $\phi_i$ can be decomposed as
 {{Equation1
-|equation=<math>\pyiphii(y_i , \phi_i ; \theta) = \pcyiphii(y_i {{!}} \phi_i ; a)\pphii( \phi_i ; \beta, \Omega).
+|equation= <math>
+\pyiphii(y_i , \phi_i ; \theta) = \pcyiphii(y_i {{!}} \phi_i ; a)\pphii( \phi_i ; \beta, \Omega).
 </math> }}
+<br>
-. Assume now that some components of $\phi_i$ have no inter-individual variability. More precisely, let $\phi_i=(\phi_i^{(1)},\phi_i^{(0)})$ and $\beta=(\beta_1,\beta_0)$ such that
+<li>Suppose that some components of $\phi_i$ have no inter-individual variability. More precisely, let $\phi_i=(\phi_i^{(1)} \phi_i^{(0)})$ and $\beta=(\beta_1,\beta_0)$, such that</li>
 {{Equation1
 |equation=<math>\begin{eqnarray}
 \phi_i^{(1)} &\sim& {\cal N}(\beta_1, \Omega_1) \\
-\phi_i^{(0)} &=& \beta_0
+\phi_i^{(0)} &=& \beta_0 ,
 \end{eqnarray}</math> }}
-:where $\Omega_1$ is positive definite. Here, $\theta=(\beta_1,\beta_0,\Omega,a)$ and
+and $\Omega_1$ is positive definite. Here, $\theta=(\beta_1,\beta_0,\Omega_1,a)$ and
 {{Equation1
@@ Line 142: / Line 103: @@
+<br>
-. Assume that $\phi_i = (\phi_{i,1}, \phi_{i,2})$ such that
+<li>Assume instead that $\phi_i = (\phi_{i,1}, \phi_{i,2})$, where</li>
 {{Equation1
 |equation=<math>\begin{eqnarray}
 \phi_{i,1} &=& \beta_1 + \omega_1\eta_i \\
-\phi_{i,2} &=& \beta_2 + \omega_2\eta_i
+\phi_{i,2} &=& \beta_2 + \omega_2\eta_i ,
 \end{eqnarray}</math> }}
-:where $\eta_i \sim {\cal N}(0,1)$. A regular model is the joint distribution of $y_i$ and $\eta_i$. One can then use the following $\eta$-representation for instance:
+and $\eta_i \sim {\cal N}(0,1)$. Here, the useful model is the joint distribution of $y_i$ and $\eta_i$. We can  use for instance the following $\eta$-representation:
 {{Equation1
-|equation=<math>\pyietai(y_i , \eta_i ; \theta) = \pcyietai(y_i {{!}} \eta_i ;\theta)\petai( \eta_i).
+|equation=<math>
+\pyietai(y_i , \eta_i ; \theta) = \pcyietai(y_i {{!}} \eta_i ;\theta)\petai( \eta_i),
 </math> }}
-:where $\theta= (\beta_1,\beta_2, \omega_1,\omega_2,a)$.
+: where $\theta= (\beta_1,\beta_2, \omega_1,\omega_2,a)$.
+</ol>
 }}
 <br>
+== Some notation ==
-== Some notations ==
+We assume that the set of population parameters $\theta$ takes its values in $\Theta$, an open subset of $\Rset^m$.
-Let $\theta$ be the set of population parameters. We assume that $\theta$ takes its values in $\Theta$, an open subset of $\Rset^m$.
-Let $f : \Theta \to \Rset$ be a twice differentiable function of $\theta$. We will denote $\Dt{f(\theta)} = (\partial f(\theta)/\partial \theta_j, 1 \leq j \leq m) $  the gradient of $f$ (i.e. the vector of partial derivatives of $f$) and $\DDt{f(\theta)} = = (\partial^2 f(\theta)/\partial \theta_j\partial \theta_k, 1 \leq j,k \leq m) $  the Hessian  of $f$ (i.e.  the square matrix of second-order partial derivatives of $f$).
+Let $f : \Theta \to \Rset$ be a twice differentiable function of $\theta$. We will denote $\Dt{f(\theta)} = (\partial f(\theta)/\partial \theta_j, 1 \leq j \leq m) $  the gradient of $f$ (i.e., the vector of partial derivatives of $f$) and $\DDt{f(\theta)} = (\partial^2 f(\theta)/\partial \theta_j\partial \theta_k, 1 \leq j,k \leq m) $  the Hessian  of $f$ (i.e.,  the square matrix of second-order partial derivatives of $f$).
-</div>
+{{Next
-<!-- %\begin{eqnarray} -->
+|link=The SAEM algorithm for estimating population parameters }}
-<!--%y_i | \psi_i &\sim& \pcyipsii(y_i | \psi_i) \\ -->
-<!--%h(\psi_i) &\sim_{i.i.d}& {\cal N}( \mu , \Omega), -->
-<!--%\end{eqnarray} -->
-<!--%where $h(\psi_i)=(h_1(\psi_{i,1}), h_2(\psi_{i,2}), \ldots , h_d(\psi_{i,d}) )^\prime$ is a $d$-vector of (transformed) individual parameters, $\mu$ a $d$-vector of fixed effects and  $\Omega$ a $d\times d$ variance-covariance matrix. -->

Difference between revisions of "Introduction and notation"

Latest revision as of 13:54, 7 June 2013

Different representations of the same model

Some notation

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

WikiPopix

Introduction

Models

Tasks & Tools

Methods

Download files

Tools