---
title: "Probability Distributions"
author: "Vladimír Holý"
date: "`r Sys.Date()`"
output:
  rmarkdown::html_vignette:
    toc: true
    toc_depth: 2
vignette: >
  %\VignetteIndexEntry{Probability Distributions}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

# Binary Data

## Bernoulli Distribution

### Probabilistic Parametrization

#### Parameter

- Probability parameter $p \in (0, 1)$

#### Probability Mass Function

$$
\begin{aligned}
\mathrm{P} [Y = y | p] &= \begin{cases}
1 - p & \text{ for } y = 0 \\
p & \text{ for } y = 1 \\
\end{cases} \\
\end{aligned}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= p \\
\mathrm{var}[Y] &= p (1 - p) \\
\end{aligned}
$$

#### Score

$$
\nabla_{m} (y; p) = \begin{cases}
\frac{1}{p - 1} & \text{ for } y = 0 \\
\frac{1}{p} & \text{ for } y = 1 \\
\end{cases}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{p, p} (p) &= \frac{1}{p (1 - p)} \\
\end{aligned}
$$

# Categorical Data

## Categorical Distribution

### Worth Parametrization

#### Parameters

- Worth parameters $w_i \in (0, \infty), i = 1, \ldots, n$

#### Vector Notation

- Worth vector $\boldsymbol{w}$ of length $n$

#### Probability Mass Function

$$
\begin{aligned}
\mathrm{P} [\boldsymbol{Y} = \boldsymbol{y} | \boldsymbol{w}] &= \frac{1}{\sum_{i=1}^n w_i} \prod_{i=1}^n w_i^{y_i}
\end{aligned}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[\boldsymbol{Y}] &= \frac{1}{\sum_{i=1}^n w_i} \boldsymbol{w} \\
\mathrm{var}[\boldsymbol{Y}] &= \frac{1}{\sum_{i=1}^n w_i}  \mathrm{diag} (\boldsymbol{w}) - \frac{1}{\left( \sum_{i=1}^n w_i \right)^2} \boldsymbol{w} \boldsymbol{w}^\intercal \\
\end{aligned}
$$

#### Score

$$
\nabla_{\boldsymbol{w}} (\boldsymbol{y}; \boldsymbol{w}) = \boldsymbol{y} \oslash \boldsymbol{w} - \frac{1}{\sum_{i=1}^n w_i} \boldsymbol{1}_n
$$

#### Fisher Information

$$
\mathcal{I}_{\boldsymbol{w}, \boldsymbol{w}} (\boldsymbol{w}) = \mathrm{diag} \left( \sum_{i=1}^n w_i \boldsymbol{1}_n  \oslash \boldsymbol{w} \right) - \frac{1}{\left( \sum_{i=1}^n w_i \right)^2} \boldsymbol{1}_{n \times n}
$$

### Notes

- We treat the categorical distribution as a multivariate distribution. For $n$ categories, observations are in the form of vectors of length $n$ with exactly one element equal to 1 and the others to 0.

- The probability mass function is invariant to the multiplication by a constant of the worth parameters. In the case of the logarithmic transformation, it is invariant to the addition of a constant to the transformed worth parameters. The parameters therefore need to be standardized, e.g. to zero sum in the latter case.

# Ranking Data

## Plackett–Luce Distribution

### Worth Parametrization

#### Parameters

- Worth parameters $w_i \in (0, \infty), i = 1, \ldots, n$

#### Ranking Notation

- Worth parameters by rank $w_{j^{\mathrm{th}}}, j = 1, \ldots, n$

#### Probability Mass Function

$$
\mathrm{P} [\boldsymbol{Y} = \boldsymbol{y} | w_1, \ldots, w_n] = \prod_{j=1}^n \frac{w_{j^{\mathrm{th}}}}{\sum_{k=j}^n w_{k^{\mathrm{th}}}}
$$

#### Score

$$
\nabla_{w_i} (\boldsymbol{y}; w_1, \ldots, w_n) = \frac{1}{w_i} - \sum_{j=1}^{y_i} \frac{1}{\sum_{k = j}^n w_{k^{\mathrm{th}}}}
$$

### Notes

- The expected value, the variance, and the Fisher information are computed directly from the definitions as sums over all possible rankings. As the number of permutations grows drastically with increasing $n$, we only use this approach for $n \leq 6$. For $n \geq 7$, we randomly sample 1 000 rankings. We locally set seed so the results are always the same.

- The probability mass function is invariant to the multiplication by a constant of the worth parameters. In the case of the logarithmic transformation, it is invariant to the addition of a constant to the transformed worth parameters. The parameters therefore need to be standardized, e.g. to zero sum in the latter case.

### Further Reading

- Alvo, M. and Yu, P. L. H. (2014). *Statistical Methods for Ranking Data*. Springer. doi: [10.1007/978-1-4939-1471-5](https://doi.org/10.1007/978-1-4939-1471-5).

- Holý, V. and Zouhar, J. (2022). Modelling Time-Varying Rankings with Autoregressive and Score-Driven Dynamics. Journal of the Royal Statistical Society: Series C (Applied Statistics), **71**(5). doi: [10.1111/rssc.12584](https://doi.org/10.1111/rssc.12584).

- Luce, R. D. (1977). The Choice Axiom after Twenty Years. *Journal of Mathematical Psychology*, **15**(3), 215–233. doi: [10.1016/0022-2496(77)90032-3](https://doi.org/10.1016/0022-2496(77)90032-3).

- Plackett, R. L. (1975). The Analysis of Permutations. *Journal of the Royal Statistical Society: Series C (Applied Statistics)*, **24**(2), 193–202. doi: [10.2307/2346567](https://doi.org/10.2307/2346567).

# Count Data

## Double Poisson Distribution

### Mean Parametrization

#### Parameters

- Mean parameter $m \in (0, \infty)$
- Dispersion parameter $s \in (0, \infty)$

#### Probability Mass Function

$$
\mathrm{P} [Y = y | m, s] \approx \frac{1}{1 + \frac{1 - s}{12 s m} \left(1 + \frac{1}{s m} \right)} \sqrt{s} \frac{y^y}{y!} \left( \frac{m}{y} \right)^{s y} \exp(s y - s m - y)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &\approx m \\
\mathrm{var}[Y] &\approx \frac{m}{s} \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s) &\approx \frac{s}{m} (y - m) \\
\nabla_{s} (y; m, s) &\approx \frac{1}{2 s} - m \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s) &\approx \frac{s}{m} \\
\mathcal{I}_{m, s} (m, s) &\approx 0 \\
\mathcal{I}_{s, s} (m, s) &\approx \frac{1}{2 s^2} \\
\end{aligned}
$$

### Note

- The probability mass function is not available in a closed form. We use the approximation of Efron (1986) for the probability mass function, the mean, the variance, the score, and the Fisher information.

### Further Reading

- Aragon, D. C., Achcar, J. A., and Martinez, E. Z. (2018). Maximum Likelihood and Bayesian Estimators for the Double Poisson Distribution. *Journal of Statistical Theory and Practice*, **12**(4), 886–911. doi: [10.1080/15598608.2018.1489919](https://doi.org/10.1080/15598608.2018.1489919).

- Cameron, A. C. and Trivedi, P. K. (2013). *Regression Analysis of Count Data*. Second Edition. Cambridge University Press. doi: [10.1017/cbo9781139013567](https://doi.org/10.1017/cbo9781139013567).

- Efron, B. (1986). Double Exponential Families and Their Use in Generalized Linear Regression. *Journal of the American Statistical Association*, **81**(395), 709–721. doi: [10.1080/01621459.1986.10478327](https://doi.org/10.1080/01621459.1986.10478327).

- Hilbe, J. M. (2011). *Negative Binomial Regression*. Second Edition. Cambridge University Press. doi: [10.1017/cbo9780511973420](https://doi.org/10.1017/cbo9780511973420).

- Holý, V. and Tomanová, P. (2022). Modeling Price Clustering in High-Frequency Prices. *Quantitative Finance*. doi: [10.1080/14697688.2022.2050285](https://doi.org/10.1080/14697688.2022.2050285).

- Sellers, K. F. and Morris, D. S. (2017). Underdispersion Models: Models That Are “Under the Radar.” *Communications in Statistics - Theory and Methods*, **46**(24), 12075–12086. doi: [10.1080/03610926.2017.1291976](https://doi.org/10.1080/03610926.2017.1291976).

- Zou, Y., Geedipally, S. R., and Lord, D. (2013). Evaluating the Double Poisson Generalized Linear Model. *Accident Analysis and Prevention*, **59**, 497–505. doi: [10.1016/j.aap.2013.07.017](https://doi.org/10.1016/j.aap.2013.07.017).

## Geometric Distribution

### Mean Parametrization

#### Parameter

- Mean parameter $m \in (0, \infty)$

#### Probability Mass Function

$$
\mathrm{P} [Y = y | m] = \frac{1}{1 + m} \left( \frac{m}{1 + m} \right)^{y}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m \\
\mathrm{var}[Y] &= m (1 + m) \\
\end{aligned}
$$

#### Score

$$
\nabla_{m} (y; m) = \frac{y - m}{m (1 + m) }
$$

#### Fisher Information

$$
\mathcal{I}_{m, m} (m) = \frac{1}{m (1 + m)}
$$

### Probabilistic Parametrization

#### Parameter

- Probability parameter $p \in (0, 1)$

#### Probability Mass Function

$$
\mathrm{P} [Y = y | p] = p (1 - p)^{y}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= \frac{1 - p}{p} \\
\mathrm{var}[Y] &= \frac{1 - p}{p^2} \\
\end{aligned}
$$

#### Score

$$
\nabla_{p} (y; p) = \frac{p y + p - 1}{p (p - 1)}
$$

#### Fisher Information

$$
\mathcal{I}_{p, p} (p) = \frac{1}{p^2 (1 - p)}
$$

## Negative Binomial Distribution

### NB2 Parametrization

#### Parameters

- Mean parameter $m \in (0, \infty)$
- Dispersion parameter $s \in (0, \infty)$

#### Probability Mass Function

$$
\mathrm{P} [Y = y | m, s] = \frac{\Gamma (y + s^{-1})}{\Gamma (y + 1) \Gamma (s^{-1})} \left( \frac{1}{1 + s m} \right)^{s^{-1}} \left( \frac{s m}{1 + s m} \right)^{y}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m \\
\mathrm{var}[Y] &= m (1 + s m) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s) &= \frac{y - m}{m (1 + s m) } \\
\nabla_{s} (y; m, s) &= \frac{ y - m}{s (1 + s m)} + \frac{1}{s^2} \left( \ln(1 + s m) + \psi_0 \left( \frac{1}{s} \right) - \psi_0 \left( y + \frac{1}{s} \right) \right) \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s) &= \frac{1}{m (1 + s m)} \\
\mathcal{I}_{m, s} (m, s) &= 0 \\
\mathcal{I}_{s, s} (m, s) &\approx \frac{1}{s^4} \left( \ln(1 + s m) + \psi_0 \left( \frac{1}{s} \right) - \psi_0 \left( m + \frac{1}{s} \right) \right)^2 \\
\end{aligned}
$$

### Probabilistic Parametrization

#### Parameters

- Probability parameter $p \in (0, 1)$
- Size parameter $r \in (0, \infty)$

#### Probability Mass Function

$$
\mathrm{P} [Y = y | p, r] = \frac{\Gamma(y + r)}{\Gamma(y + 1) \Gamma(r)} (1 - p)^y p^r
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= \frac{r (1 - p)}{p} \\
\mathrm{var}[Y] &= \frac{r (1 - p)}{p^2} \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{p} (y; p, r) &= \frac{p r + p  y - r}{p (p - 1)} \\
\nabla_{r} (y; p, r) &= \ln(p) - \psi_0(r) + \psi_0(y + r) \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{p, p} (p, r) &= \frac{r}{p^2 (1 - p)} \\
\mathcal{I}_{p, r} (p, r) &= -\frac{1}{p} \\
\mathcal{I}_{r, r} (p, r) &\approx \left( \ln(p) - \psi_0(r) + \psi_0 \left( \frac{r}{p} \right) \right)^2 \\
\end{aligned}
$$

### Note

- The Fisher information for the dispersion or size parameter, $\mathcal{I}_{s, s} (m, s)$ or $\mathcal{I}_{r, r} (p, r)$, is not available in a closed form. To speed up calculations, we use a rough approximation by replacing $y$ with its expected value.

### Further Reading

- Cameron, A. C. and Trivedi, P. K. (1986). Econometric Models Based on Count Data: Comparisons and Applications of Some Estimators and Tests. *Journal of Applied Econometrics*, **1**(1), 29–53. doi: [10.1002/jae.3950010104](https://doi.org/10.1002/jae.3950010104).

- Cameron, A. C. and Trivedi, P. K. (2013). *Regression Analysis of Count Data*. Second Edition. Cambridge University Press. doi: [10.1017/cbo9781139013567](https://doi.org/10.1017/cbo9781139013567).

- Hilbe, J. M. (2011). *Negative Binomial Regression*. Second Edition. Cambridge University Press. doi: [10.1017/cbo9780511973420](https://doi.org/10.1017/cbo9780511973420).

## Poisson Distribution

### Mean Parametrization

#### Parameter

- Mean parameter $m \in (0, \infty)$

#### Probability Mass Function

$$
\mathrm{P} [Y = y | m] = \frac{m^y}{y!} \exp(-m)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m \\
\mathrm{var}[Y] &= m \\
\end{aligned}
$$

#### Score

$$
\nabla_{m} (y; m) = \frac{y - m}{m}
$$

#### Fisher Information

$$
\mathcal{I}_{m, m} (m) = \frac{1}{m}
$$

### Further Reading

- Cameron, A. C. and Trivedi, P. K. (2013). *Regression Analysis of Count Data*. Second Edition. Cambridge University Press. doi: [10.1017/cbo9781139013567](https://doi.org/10.1017/cbo9781139013567).

- Davis, R. A., Dunsmuir, W. T. M., and Street, S. B. (2003). Observation-Driven Models for Poisson Counts. *Biometrika*, **90**(4), 777–790. doi: [10.1093/biomet/90.4.777](https://doi.org/10.1093/biomet/90.4.777).

- Hilbe, J. M. (2011). *Negative Binomial Regression*. Second Edition. Cambridge University Press. doi: [10.1017/cbo9780511973420](https://doi.org/10.1017/cbo9780511973420).

## Zero-Inflated Geometric Distribution

#### Parameters

- Mean parameter $m \in (0, \infty)$
- Zero inflation parameter $p \in (0, 1)$

#### Probability Mass Function

$$
\begin{aligned}
\mathrm{P} [Y = y | m, p] &= \begin{cases}
p + (1 - p)  \left( \frac{1}{1 + m} \right) & \text{ for } y = 0 \\
(1 - p) \left( \frac{1}{1 + m} \right) \left( \frac{m}{1 + m} \right)^{y} & \text{ for } y \geq 1 \\
\end{cases} \\
\end{aligned}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m (1 - p) \\
\mathrm{var}[Y] &= m(1 - p) (1 + p m + m) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, p) &= \begin{cases}
\frac{p - 1}{(1 + m) (1 + p m)} & \text{ for } y = 0 \\
\frac{y - m}{m (1 + m) } & \text{ for } y \geq 1 \\
\end{cases} \\
\nabla_{p} (y; m, p) &= \begin{cases}
\frac{m}{1 + p m} & \text{ for } y = 0 \\
\frac{1}{p - 1} & \text{ for } y \geq 1 \\
\end{cases} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, p) &= \frac{(1 - p) (1 + m + p m^2)}{m (1 + m) (1 + p m)} \\
\mathcal{I}_{m, p} (m, p) &= - \frac{1}{ (1 + m) ( 1 + p m) } \\
\mathcal{I}_{p, p} (m, p) &= \frac{m}{(1 - p) ( 1 + p m)} \\
\end{aligned}
$$

### Further Reading

- Blasques, F., Holý, V., and Tomanová, P. (2022). Zero-Inflated Autoregressive Conditional Duration Model for Discrete Trade Durations with Excessive Zeros. Working Paper. arXiv: [1812.07318](https://arxiv.org/abs/1812.07318).

- Cameron, A. C. and Trivedi, P. K. (2013). *Regression Analysis of Count Data*. Second Edition. Cambridge University Press. doi: [10.1017/cbo9781139013567](https://doi.org/10.1017/cbo9781139013567).

- Greene, W. H. (1994). Accounting for Excess Zeros and Sample Selection in Poisson and Negative Binomial Regression Models. *NYU Stern School of Business Research Paper Series*, EC-94-10. SSRN: [1293115](https://www.ssrn.com/abstract=1293115).

- Hilbe, J. M. (2011). *Negative Binomial Regression*. Second Edition. Cambridge University Press. doi: [10.1017/cbo9780511973420](https://doi.org/10.1017/cbo9780511973420).

- Lambert, D. (1992). Zero-Inflated Poisson Regression, with an Application to Defects in Manufacturing. *Technometrics*, **34**(1), 1–14. doi: [10.2307/1269547](https://doi.org/10.2307/1269547).

## Zero-Inflated Negative Binomial Distribution

### NB2 Parametrization

#### Parameters

- Mean parameter $m \in (0, \infty)$
- Dispersion parameter $s \in (0, \infty)$
- Zero inflation parameter $p \in (0, 1)$

#### Probability Mass Function

$$
\begin{aligned}
\mathrm{P} [Y = y | m, s, p] &= \begin{cases}
p + (1 - p)  \left( \frac{1}{1 + s m} \right)^{s^{-1}} & \text{ for } y = 0 \\
(1 - p) \frac{\Gamma (y + s^{-1})}{\Gamma (y + 1) \Gamma (s^{-1})} \left( \frac{1}{1 + s m} \right)^{s^{-1}} \left( \frac{s m}{1 + s m} \right)^{y} & \text{ for } y \geq 1 \\
\end{cases} \\
\end{aligned}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m (1 - p) \\
\mathrm{var}[Y] &= m(1 - p) (1 + p m + s m) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s, p) &= \begin{cases}
\frac{p - 1}{(1 + s m) \left( 1 + p (1 + s m)^{s^{-1}} - p \right)} & \text{ for } y = 0 \\
\frac{y - m}{m (1 + s m) } & \text{ for } y \geq 1 \\
\end{cases} \\
\nabla_{s} (y; m, s, p) &= \begin{cases}
\frac{(1 - p) \left( (1 +  s m) \ln(1 + s m) -s m \right) }{  s^2 (1 + s m) \left( 1 + p (1 + s m)^{s^{-1}}- p \right) } & \text{ for } y = 0 \\
\frac{ s (y - m) + (1 + s m) \left( \ln(1 + s m) + \psi_0 \left( s^{-1} \right) - \psi_0 \left( y + s^{-1} \right) \right) }{s^2 (1 + s m)} & \text{ for } y \geq 1 \\
\end{cases} \\
\nabla_{p} (y; m, s, p) &= \begin{cases}
\frac{(1 + s m)^{s^{-1}} - 1}{1 + p (1 + s m)^{s^{-1}}- p} & \text{ for } y = 0 \\
\frac{1}{p - 1} & \text{ for } y \geq 1 \\
\end{cases} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s, p) &= \frac{p(p - 1)}{(1 + s m)^2 \left( 1 +  p (1 + s m)^{s^{-1}} - p \right)} + \frac{1 -p}{m(1 + s m)} \\
\mathcal{I}_{m, s} (m, s, p) &= \frac{\left( p - p^2 \right) \left( (1 + s m) \ln(1 + s m) - s m \right) }{s^2 (1 + s m)^2 \left( 1 + p (1 + s m)^{s^{-1}} -p \right)} \\
\mathcal{I}_{m, p} (m, s, p) &= \frac{-1}{ (1 + s m) \left( 1 + p (1 + s m)^{s^{-1}} - p \right) }\\
\mathcal{I}_{s, s} (m, s, p) &\approx \frac{1}{s^4} \left( \ln(1 + s m) + \psi_0 \left( s^{-1} \right) - \psi_0 \left( y + s^{-1} \right) \right)^2 \left( 1 - p - (1 - p)  \left( 1 + s m \right)^{-s^{-1}} \right) \\
& \qquad + \frac{(1 - p)^2 \left( (1 + s m) \ln(1 + s m) - s m \right)^2}
{s^4 (1 + s m)^{2 + s^{-1}} \left( 1 + p (1 + s m)^{s^{-1}} - p \right)} \\
\mathcal{I}_{s, p} (m, s, p) &= \frac{(1 + s m) \ln(1 + s m) - s m}{s^2 (1 + s m) \left( 1 + p (1 + s m)^{s^{-1}} - p \right)} \\
\mathcal{I}_{p, p} (m, s, p) &= \frac{1 - (1 + s m)^{s^{-1}}}{(p - 1) \left( 1 + p (1 + s m)^{s^{-1}} - p \right)}
\end{aligned}
$$

### Note

- The Fisher information for the dispersion parameter, $\mathcal{I}_{s, s} (m, s, p)$, is not available in a closed form. To speed up calculations, we use an approximation by replacing $y$ with its expected value combined with the zero value.

### Further Reading

- Blasques, F., Holý, V., and Tomanová, P. (2022). Zero-Inflated Autoregressive Conditional Duration Model for Discrete Trade Durations with Excessive Zeros. Working Paper. arXiv: [1812.07318](https://arxiv.org/abs/1812.07318).

- Cameron, A. C. and Trivedi, P. K. (2013). *Regression Analysis of Count Data*. Second Edition. Cambridge University Press. doi: [10.1017/cbo9781139013567](https://doi.org/10.1017/cbo9781139013567).

- Greene, W. H. (1994). Accounting for Excess Zeros and Sample Selection in Poisson and Negative Binomial Regression Models. *NYU Stern School of Business Research Paper Series*, EC-94-10. SSRN: [1293115](https://www.ssrn.com/abstract=1293115).

- Hilbe, J. M. (2011). *Negative Binomial Regression*. Second Edition. Cambridge University Press. doi: [10.1017/cbo9780511973420](https://doi.org/10.1017/cbo9780511973420).

- Lambert, D. (1992). Zero-Inflated Poisson Regression, with an Application to Defects in Manufacturing. *Technometrics*, **34**(1), 1–14. doi: [10.2307/1269547](https://doi.org/10.2307/1269547).

## Zero-Inflated Poisson Distribution

### Mean Parametrization

#### Parameters

- Mean parameter $m \in (0, \infty)$
- Zero inflation parameter $p \in (0, 1)$

#### Probability Mass Function

$$
\begin{aligned}
\mathrm{P} [Y = y | m, p] &= \begin{cases}
p + (1 - p)  \exp(-m) & \text{ for } y = 0 \\
(1 - p) \frac{m^y}{y!} \exp(-m) & \text{ for } y \geq 1 \\
\end{cases} \\
\end{aligned}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m (1 - p) \\
\mathrm{var}[Y] &= m(1 - p) (1 + p m) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s, p) &= \begin{cases}
\frac{p - 1}{p \exp(m) - p + 1} & \text{ for } y = 0 \\
\frac{y - m}{m} & \text{ for } y \geq 1 \\
\end{cases} \\
\nabla_{p} (y; m, s, p) &= \begin{cases}
\frac{\exp(m) - 1}{p \exp(m) - p + 1} & \text{ for } y = 0 \\
\frac{1}{p - 1} & \text{ for } y \geq 1 \\
\end{cases} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s, p) &= \frac{p (p - 1)}{p \exp(m) - p + 1} - \frac{p - 1}{m} \\
\mathcal{I}_{m, p} (m, s, p) &= - \frac{1}{p \exp(m) - p + 1} \\
\mathcal{I}_{p, p} (m, s, p) &= \frac{\exp(m) - 1}{(1 - p) (p \exp(m) - p + 1)} \\
\end{aligned}
$$

### Note

- The Fisher information for the dispersion parameter, $\mathcal{I}_{s, s} (m, s, p)$, is not available in a closed form. To speed up calculations, we use an approximation by replacing $y$ with its expected value.

### Further Reading

- Blasques, F., Holý, V., and Tomanová, P. (2022). Zero-Inflated Autoregressive Conditional Duration Model for Discrete Trade Durations with Excessive Zeros. Working Paper. arXiv: [1812.07318](https://arxiv.org/abs/1812.07318).

- Cameron, A. C. and Trivedi, P. K. (2013). *Regression Analysis of Count Data*. Second Edition. Cambridge University Press. doi: [10.1017/cbo9781139013567](https://doi.org/10.1017/cbo9781139013567).

- Greene, W. H. (1994). Accounting for Excess Zeros and Sample Selection in Poisson and Negative Binomial Regression Models. *NYU Stern School of Business Research Paper Series*, EC-94-10. SSRN: [1293115](https://www.ssrn.com/abstract=1293115).

- Hilbe, J. M. (2011). *Negative Binomial Regression*. Second Edition. Cambridge University Press. doi: [10.1017/cbo9780511973420](https://doi.org/10.1017/cbo9780511973420).

- Lambert, D. (1992). Zero-Inflated Poisson Regression, with an Application to Defects in Manufacturing. *Technometrics*, **34**(1), 1–14. doi: [10.2307/1269547](https://doi.org/10.2307/1269547).

# Integer Data

## Skellam Distribution

### Difference Parametrization

#### Parameters

- First rate parameter $r_1 \in (0, \infty)$
- Second rate parameter $r_2 \in (0, \infty)$

#### Probability Mass Function

$$
\begin{aligned}
\mathrm{P} [Y = y | r_1, r_2] &= \exp(-r_1 - r_2) \left( \frac{r_1}{r_2} \right)^{\frac{y}{2}} I_y \left( 2 \sqrt{r_1  r_2} \right)
\end{aligned}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= r_1 - r_2 \\
\mathrm{var}[Y] &= r_1 + r_2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{r_1} (y; r_1, r_2) &= \sqrt{\frac{r_2}{r_1}} \frac{I_{y-1} \left( 2 \sqrt{r_1 r_2} \right)}{I_y \left( 2 \sqrt{r_1 r_2} \right) } - 1 \\
\nabla_{r_2} (y; r_1, r_2) &= \sqrt{\frac{r_1}{r_2}} \frac{I_{y-1} \left( 2 \sqrt{r_1 r_2} \right)}{I_y \left( 2 \sqrt{r_1 r_2} \right) } -\frac{y}{r_2} - 1 \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{r_1, r_1} (r_1, r_2) &\approx  \frac{r_2}{r_1} \left( \frac{I_{r_1 - r_2 - 1} \left(2 \sqrt{r_1 r_2} \right) }{I_{r_1 - r_2} \left(2 \sqrt{r_1 r_2} \right) } \right)^2 - 2 \sqrt{\frac{r_2}{r_1}} \frac{I_{r_1 - r_2 - 1} \left(2 \sqrt{r_1 r_2} \right) }{I_{r_1 - r_2} \left(2 \sqrt{r_1 r_2} \right) } + 1 \\
\mathcal{I}_{r_1, r_2} (r_1, r_2) &\approx  \left( \frac{I_{r_1 - r_2 - 1} \left(2 \sqrt{r_1 r_2} \right) }{I_{r_1 - r_2} \left(2 \sqrt{r_1 r_2} \right) } \right)^2 - 2 \sqrt{\frac{r_1}{r_2}} \frac{I_{r_1 - r_2 - 1} \left(2 \sqrt{r_1 r_2} \right) }{I_{r_1 - r_2} \left(2 \sqrt{r_1 r_2} \right) } + \frac{r_1}{r_2} \\
\mathcal{I}_{r_2, r_2} (r_1, r_2) &\approx \frac{r_1}{r_2} \left( \frac{I_{r_1 - r_2 - 1} \left(2 \sqrt{r_1 r_2} \right) }{I_{r_1 - r_2} \left(2 \sqrt{r_1 r_2} \right) } \right)^2 - 2 \left( \frac{r_1}{r_2} \right)^{\frac{3}{2}} \frac{I_{r_1 - r_2 - 1} \left(2 \sqrt{r_1 r_2} \right) }{I_{r_1 - r_2} \left(2 \sqrt{r_1 r_2} \right) } + \left( \frac{r_1}{r_2} \right)^2 \\
\end{aligned}
$$

### Mean-Dispersion Parametrization

#### Parameters

- Mean parameter $m \in \mathbb{R}$
- Dispersion parameter $s \in (0, \infty)$

#### Probability Mass Function

$$
\mathrm{P} [Y = y | m, s] = \exp(-|m| - s) \left( \frac{|m| + m + s}{|m| - m + s} \right)^{\frac{y}{2}} I_y \left( \sqrt{s^2 + 2 |m| s} \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m \\
\mathrm{var}[Y] &= |m| + s \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s) &= \frac{y}{2|m| + s} + \frac{\mathrm{sgn}(m) s}{2 \sqrt{s^2 + 2 |m| s}} \frac{ I_{y-1} \left( \sqrt{s^2 + 2 |m| s} \right) + I_{y+1} \left( \sqrt{s^2 + 2 |m| s} \right) }{ I_y \left( \sqrt{s^2 + 2 |m| s} \right) } - \mathrm{sgn}(m) \\
\nabla_{s} (y; m, s) &= - \frac{m y}{s^2 + 2 |m| s} + \frac{|m| + s}{2 \sqrt{s^2 + 2 |m| s}} \frac{ I_{y-1} \left( \sqrt{s^2 + 2 |m| s} \right) + I_{y+1} \left( \sqrt{s^2 + 2 |m| s} \right) }{ I_y \left( \sqrt{s^2 + 2 |m| s} \right) } - 1 \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s) &\approx \frac{s^2}{4 \left( s^2 + 2|m|s \right)} \left( \frac{2 (|m| + s)}{\sqrt{s^2 + 2 |m| s}} - \frac{ I_{m-1} \left( \sqrt{s^2 + 2 |m| s} \right) + I_{m+1} \left( \sqrt{s^2 + 2 |m| s} \right) }{ I_m \left( \sqrt{s^2 + 2 |m| s} \right)} \right)^2 \\
\mathcal{I}_{m, s} (m, s) &\approx \frac{\mathrm{sgn}(m) (|m| + s) s}{4 \left( s^2 + 2|m|s \right)} \left( \frac{2 (|m| + s)}{\sqrt{s^2 + 2 |m| s}} - \frac{ I_{m-1} \left( \sqrt{s^2 + 2 |m| s} \right) + I_{m+1} \left( \sqrt{s^2 + 2 |m| s} \right) }{ I_m \left( \sqrt{s^2 + 2 |m| s} \right)} \right)^2 \\
\mathcal{I}_{s, s} (m, s) &\approx \frac{(|m| + s)^2}{4 \left( s^2 + 2|m|s \right)} \left( \frac{2 (|m| + s)}{\sqrt{s^2 + 2 |m| s}} - \frac{ I_{m-1} \left( \sqrt{s^2 + 2 |m| s} \right) + I_{m+1} \left( \sqrt{s^2 + 2 |m| s} \right) }{ I_m \left( \sqrt{s^2 + 2 |m| s} \right)} \right)^2 \\
\end{aligned}
$$

### Mean-Variance Parametrization

#### Parameters

- Mean parameter $m \in \mathbb{R}$
- Variance parameter $s \in (|m|, \infty)$

#### Probability Mass Function

$$
\mathrm{P} [Y = y | m, s] = \exp(-s) \left( \frac{s + m}{s - m} \right)^{\frac{y}{2}} I_y \left( \sqrt{s^2 - m^2} \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m \\
\mathrm{var}[Y] &= s \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s) &= \frac{s y}{s^2 - m^2} - \frac{m}{2 \sqrt{s^2 - m^2}} \frac{ I_{y-1} \left( \sqrt{s^2 - m^2} \right) + I_{y+1} \left( \sqrt{s^2 - m^2} \right) }{ I_y \left( \sqrt{s^2 - m^2} \right) } \\
\nabla_{s} (y; m, s) &= -\frac{m y}{s^2 - m^2} + \frac{s}{2 \sqrt{s^2 - m^2}} \frac{ I_{y-1} \left( \sqrt{s^2 - m^2} \right) + I_{y+1} \left( \sqrt{s^2 - m^2} \right) }{ I_y \left( \sqrt{s^2 - m^2} \right) } - 1\\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s) &\approx \frac{m^2}{4 \left( s^2 - m^2 \right)} \left( \frac{2 s}{\sqrt{s^2 - m^2}} - \frac{ I_{m-1} \left( \sqrt{s^2 - m^2} \right) + I_{m+1} \left( \sqrt{s^2 - m^2} \right) }{ I_m \left( \sqrt{s^2 - m^2} \right) } \right)^2 \\
\mathcal{I}_{m, s} (m, s) &\approx - \frac{m s}{4 \left( s^2 - m^2 \right)} \left( \frac{2 s}{\sqrt{s^2 - m^2}} - \frac{ I_{m-1} \left( \sqrt{s^2 - m^2} \right) + I_{m+1} \left( \sqrt{s^2 - m^2} \right) }{ I_m \left( \sqrt{s^2 - m^2} \right) } \right)^2 \\
\mathcal{I}_{s, s} (m, s) &\approx \frac{s^2}{4 \left( s^2 - m^2 \right)} \left( \frac{2 s}{\sqrt{s^2 - m^2}} - \frac{ I_{m-1} \left( \sqrt{s^2 - m^2} \right) + I_{m+1} \left( \sqrt{s^2 - m^2} \right) }{ I_m \left( \sqrt{s^2 - m^2} \right) } \right)^2 \\
\end{aligned}
$$

### Note

- The computation of the Fisher information is quite intricate and we resort to an approximation by replacing $y$ with its expected value.

### Further Reading

- Alzaid, A. A. and Omair, M. A. (2010). On the Poisson Difference Distribution Inference and Applications. *Bulletin of the Malaysian Mathematical Sciences Society*, **33**(1), 17–45. EuDML: [244475](https://eudml.org/doc/244475).

- Karlis, D. and Ntzoufras, I. (2009). Bayesian Modelling of Football Outcomes: Using the Skellam’s Distribution for the Goal Difference. *IMA Journal of Management Mathematics*, **20**(2), 133–145. doi: [10.1093/imaman/dpn026](https://doi.org/10.1093/imaman/dpn026).

- Koopman, S. J. and Lit, R. (2019). Forecasting Football Match Results in National League Competitions Using Score-Driven Time Series Models. *International Journal of Forecasting*, **35**(2), 797–809. doi: [10.1016/j.ijforecast.2018.10.011](https://doi.org/10.1016/j.ijforecast.2018.10.011).

- Koopman, S. J., Lit, R., Lucas, A., and Opschoor, A. (2018). Dynamic Discrete Copula Models for High-Frequency Stock Price Changes. *Journal of Applied Econometrics*, **33**(7), 966–985. doi: [10.1002/jae.2645](https://doi.org/10.1002/jae.2645).

- Skellam, J. G. (1946). The Frequency Distribution of the Difference Between Two Poisson Variates Belonging to Different Populations. *Journal of the Royal Statistical Society*, **109**(3), 296. doi: [10.2307/2981372](https://doi.org/10.2307/2981372).

## Zero-Inflated Skellam Distribution

### Difference Parametrization

#### Parameters

- First rate parameter $r_1 \in (0, \infty)$
- Second rate parameter $r_2 \in (0, \infty)$
- Inflation parameter $p \in (0, 1)$

#### Probability Mass Function

$$
\begin{aligned}
\mathrm{P} [Y = y | r_1, r_2, p] &= \begin{cases}
p + (1 - p) \exp(-r_1 - r_2) I_0 \left( 2 \sqrt{r_1  r_2} \right) & \text{ for } y = 0 \\
(1 - p) \exp(-r_1 - r_2) \left( \frac{r_1}{r_2} \right)^{\frac{y}{2}} I_y \left( 2 \sqrt{r_1  r_2} \right) & \text{ for } y \neq 0 \\
\end{cases} \\
\end{aligned}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= (1 - p) (r_1 - r_2) \\
\mathrm{var}[Y] &= (1 - p) \left( p \left( r_1 - r_2 \right)^2 + r_1 + r_2 \right) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{r_1} (y; r_1, r_2, p) &= \begin{cases}
\frac{(p - 1) \left( \sqrt{r_1 r_2} I_0 \left( 2 \sqrt{r_1 r_2} \right) - r_2 I_1 \left( 2 \sqrt{r_1 r_2} \right) \right)}{\sqrt{r_1 r_2} \left( p \exp(r_1 + r_2) + (1 - p) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right)} & \text{ for } y = 0 \\
\sqrt{\frac{r_2}{r_1}} \frac{I_{y-1} \left( 2 \sqrt{r_1 r_2} \right)}{I_y \left( 2 \sqrt{r_1 r_2} \right) } - 1 & \text{ for } y \neq 0 \\
\end{cases} \\
\nabla_{r_2} (y; r_1, r_2, p) &= \begin{cases}
\frac{(p - 1) \left( \sqrt{r_1 r_2} I_0 \left( 2 \sqrt{r_1 r_2} \right) - r_1 I_1 \left( 2 \sqrt{r_1 r_2} \right) \right)}{\sqrt{r_1 r_2} \left( p \exp(r_1 + r_2) + (1 - p) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right)} & \text{ for } y = 0 \\
\sqrt{\frac{r_1}{r_2}} \frac{I_{y-1} \left( 2 \sqrt{r_1 r_2} \right)}{I_y \left( 2 \sqrt{r_1 r_2} \right) } -\frac{y}{r_2} - 1 & \text{ for } y \neq 0 \\
\end{cases} \\
\nabla_{p} (y; r_1, r_2, p) &= \begin{cases}
\frac{\exp(r_1 + r_2) - I_0 \left( 2 \sqrt{r_1 r_2} \right)}{p \exp(r_1 + r_2) + (1 - p) I_0 \left( 2 \sqrt{r_1 r_2} \right)} & \text{ for } y = 0 \\
\frac{1}{p - 1} & \text{ for } y \neq 0 \\
\end{cases} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{r_1, r_1} (r_1, r_2, p) &\approx (1 - p) \left( 1 - \exp(-r_1 - r_2) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right) \left( 1 - \sqrt{\frac{r_2}{r_1}} \frac{I_{r_1 - r_2 -1} \left( 2 \sqrt{r_1 r_2} \right)}{I_{r_1 - r_2} \left( 2 \sqrt{r_1 r_2} \right)} \right)^2 \\
& \qquad + \frac{(1 - p)^2 \exp(-r_1 - r_2) \left( \sqrt{r_1 r_2} I_0 \left( 2 \sqrt{r_1 r_2} \right) - r_2 I_1 \left( 2 \sqrt{r_1 r_2} \right) \right)^2}{r_1 r_2 \left( p \exp(r_1 + r_2) + (1 - p) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right)} \\
\mathcal{I}_{r_1, r_2} (r_1, r_2, p) &\approx (1 - p) \left( 1 - \exp(-r_1 - r_2) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right) \left( 1 - \sqrt{\frac{r_2}{r_1}} \frac{I_{r_1 - r_2-1} \left( 2 \sqrt{r_1 r_2} \right)}{I_{r_1 - r_2} \left( 2 \sqrt{r_1 r_2} \right)} \right) \\
& \qquad \times \left( \frac{r_1}{r_2} - \sqrt{\frac{r_1}{r_2}} \frac{I_{r_1 - r_2 - 1} \left( 2 \sqrt{r_1 r_2} \right)}{I_{r_1 - r_2} \left( 2 \sqrt{r_1 r_2} \right)} \right) \\
& \qquad + \frac{(1 - p)^2 \exp(-r_1 - r_2) \left( \sqrt{r_1 r_2} I_0 \left( 2 \sqrt{r_1 r_2} \right) - r_2 I_1 \left( 2 \sqrt{r_1 r_2} \right) \right)}{r_1 r_2 \left( p \exp(r_1 + r_2) + (1 - p) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right)} \\
& \qquad \times \left( \sqrt{r_1 r_2} I_0 \left( 2 \sqrt{r_1 r_2} \right) - r_1 I_1 \left( 2 \sqrt{r_1 r_2} \right) \right) \\
\mathcal{I}_{r_1, p} (r_1, r_2, p) &= \frac{(p - 1) \left( 1 - \exp(-r_1 - r_2 ) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right)}{\sqrt{r_1 r_2} \left( p \exp(r_1 + r_2) + (1 - p) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right)} \\
& \qquad \times \left( \sqrt{r_1 r_2} I_0 \left( 2 \sqrt{r_1 r_2} \right) - r_2 I_1 \left( 2 \sqrt{r_1 r_2} \right) \right) \\
\mathcal{I}_{r_2, r_2} (r_1, r_2, p) &\approx (1 - p) \left( 1 - \exp(-r_1 - r_2) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right) \left( \frac{r_1}{r_2} - \sqrt{\frac{r_1}{r_2}} \frac{I_{r_1 - r_2 - 1} \left( 2 \sqrt{r_1 r_2} \right)}{I_{r_1 - r_2} \left( 2 \sqrt{r_1 r_2} \right)} \right)^2 \\
& \qquad + \frac{(1 - p)^2 \exp(-r_1 - r_2) \left( \sqrt{r_1 r_2} I_0 \left( 2 \sqrt{r_1 r_2} \right) - r_1 I_1 \left( 2 \sqrt{r_1 r_2} \right) \right)^2}{r_1 r_2 \left( p \exp(r_1 + r_2) + (1 - p) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right)} \\
\mathcal{I}_{r_2, p} (r_1, r_2, p) &= \frac{(p - 1) \left( 1 - \exp(-r_1 - r_2 ) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right)}{\sqrt{r_1 r_2} \left( p \exp(r_1 + r_2) + (1 - p) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right)} \\
& \qquad \times \left( \sqrt{r_1 r_2} I_0 \left( 2 \sqrt{r_1 r_2} \right) - r_1 I_1 \left( 2 \sqrt{r_1 r_2} \right) \right) \\
\mathcal{I}_{p, p} (r_1, r_2, p) &= \frac{\exp(r_1 + r_2) - I_0 \left( 2 \sqrt{r_1 r_2} \right)}{(1 - p) \left( p \exp(r_1 + r_2) + (1 - p) I_0 \left( 2 \sqrt{r_1 r_2} \right) \right)} \\
\end{aligned}
$$

### Mean-Dispersion Parametrization

#### Parameters

- Mean parameter $m \in \mathbb{R}$
- Dispersion parameter $s \in (0, \infty)$
- Inflation parameter $p \in (0, 1)$

#### Probability Mass Function

$$
\begin{aligned}
\mathrm{P} [Y = y | m, s, p] &= \begin{cases}
p + (1 - p) \exp(-|m| - s) I_0 \left( \sqrt{s^2 + 2 |m| s} \right) & \text{ for } y = 0 \\
(1 - p) \exp(-|m| - s) \left( \frac{|m| + m + s}{|m| - m + s} \right)^{\frac{y}{2}} I_y \left( \sqrt{s^2 + 2 |m| s} \right) & \text{ for } y \neq 0 \\
\end{cases} \\
\end{aligned}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= (1 - p) m \\
\mathrm{var}[Y] &= (1 - p) \left( |m| + s + p m^2 \right) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s, p) &= \begin{cases}
\frac{\mathrm{sgn}(m) (p - 1) \left( \sqrt{s^2 + 2 |m| s} I_0 \left( \sqrt{s^2 + 2 |m| s} \right) - s I_1 \left( \sqrt{s^2 + 2 |m| s} \right) \right)}{\sqrt{s^2 + 2 |m| s} \left( (1 - p) I_0 \left( \sqrt{s^2 + 2 |m| s} \right) + p \exp(|m| + s) \right)} & \text{ for } y = 0 \\
\frac{y}{2|m| + s} + \frac{\mathrm{sgn}(m) s}{2 \sqrt{s^2 + 2 |m| s}} \frac{ I_{y-1} \left( \sqrt{s^2 + 2 |m| s} \right) + I_{y+1} \left( \sqrt{s^2 + 2 |m| s} \right) }{ I_y \left( \sqrt{s^2 + 2 |m| s} \right) } - \mathrm{sgn}(m) & \text{ for } y \neq 0 \\
\end{cases} \\
\nabla_{s} (y; m, s, p) &= \begin{cases}
\frac{   (p - 1)  \left( \sqrt{s^2 + 2 |m| s} I_0 \left( \sqrt{s^2 + 2 |m| s} \right) - (|m| + s) I_1 \left( \sqrt{s^2 + 2 |m| s} \right) \right)    }{\sqrt{s^2 + 2 |m| s} \left( (1 - p) I_0 \left( \sqrt{s^2 + 2 |m| s} \right) + p \exp(|m| + s) \right)} & \text{ for } y = 0 \\
- \frac{m y}{s^2 + 2 |m| s} + \frac{|m| + s}{2 \sqrt{s^2 + 2 |m| s}} \frac{ I_{y-1} \left( \sqrt{s^2 + 2 |m| s} \right) + I_{y+1} \left( \sqrt{s^2 + 2 |m| s} \right) }{ I_y \left( \sqrt{s^2 + 2 |m| s} \right) } - 1 & \text{ for } y \neq 0 \\
\end{cases} \\
\nabla_{p} (y; m, s, p) &= \begin{cases}
\frac{\exp(|m| + s) - I_0 \left( \sqrt{s^2 + 2 |m| s} \right)}{p \exp(|m| + s) + (1 - p) I_0 \left( \sqrt{s^2 + 2 |m| s} \right)} & \text{ for } y = 0 \\
\frac{1}{p - 1} & \text{ for } y \neq 0 \\
\end{cases} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s, p) &\approx \frac{s^2 (1 - p) \left( 1 - \exp(-|m|-s) I_{0} \left( \sqrt{s^2 + 2 |m| s} \right) \right)}{4 s^2 + 8 |m| s} \\
& \qquad \times \left( \frac{2 (|m| + s)}{\sqrt{s^2 + 2 |m| s}} - \frac{ I_{m-1} \left( \sqrt{s^2 + 2 |m| s} \right) + I_{m+1} \left( \sqrt{s^2 + 2 |m| s} \right) }{ I_m \left( \sqrt{s^2 + 2 |m| s} \right)} \right)^2 \\
& \qquad + \frac{(1 - p)^2 \exp(-|m| - s) }{\left( s^2 + 2 |m| s \right) \left( p \exp(|m| + s) + (1 - p) I_0 \left( \sqrt{s^2 + 2 |m| s} \right) \right)} \\
& \qquad \times \left( \sqrt{s^2 + 2 |m| s} I_0 \left( \sqrt{s^2 + 2 |m| s} \right) - s I_1 \left( \sqrt{s^2 + 2 |m| s} \right) \right)^2 \\
\mathcal{I}_{m, s} (m, s, p) &\approx
\frac{\mathrm{sgn}(m) s (1 - p) (|m| + s) \left( 1 - \exp(-|m|-s) I_{0} \left( \sqrt{s^2 + 2 |m| s} \right) \right)}{4 s^2 + 8 |m| s} \\
& \qquad \times \left( \frac{2 (|m| + s)}{\sqrt{s^2 + 2 |m| s}} - \frac{ I_{m-1} \left( \sqrt{s^2 + 2 |m| s} \right) + I_{m+1} \left( \sqrt{s^2 + 2 |m| s} \right) }{ I_m \left( \sqrt{s^2 + 2 |m| s} \right)} \right)^2 \\
& \qquad + \frac{\mathrm{sgn}(m) (1 - p)^2 \exp(-|m| - s)}{\left( s^2 + 2 |m| s \right) \left( p \exp(|m| + s) + (1 - p) I_0 \left( \sqrt{s^2 + 2 |m| s} \right) \right)} \\
& \qquad \times \left( \sqrt{s^2 + 2 |m| s} I_0 \left( \sqrt{s^2 + 2 |m| s} \right) - s I_1 \left( \sqrt{s^2 + 2 |m| s} \right) \right) \\
& \qquad \times \left( \sqrt{s^2 + 2 |m| s} I_0 \left( \sqrt{s^2 + 2 |m| s} \right) - (|m| + s) I_1 \left( \sqrt{s^2 + 2 |m| s} \right) \right) \\
\mathcal{I}_{m, p} (m, s, p) &= \frac{\mathrm{sgn}(m) (p - 1) \left( 1 - \exp(-|m| - s) I_0 \left( \sqrt{s^2 + 2 |m| s} \right) \right)}{\sqrt{s^2 + 2 |m| s} \left( p \exp(|m| + s) + (1 - p) I_0 \left( \sqrt{s^2 + 2 |m| s} \right) \right)} \\
& \qquad \times \left( \sqrt{s^2 + 2 |m| s} I_0 \left( \sqrt{s^2 + 2 |m| s} \right) - s I_1 \left( \sqrt{s^2 + 2 |m| s} \right) \right) \\
\mathcal{I}_{s, s} (m, s, p) &\approx \frac{(1 - p) (|m| + s)^2 \left( 1 - \exp(-|m|-s) I_{0} \left( \sqrt{s^2 + 2 |m| s} \right) \right)}{4 s^2  + 8 |m| s} \\
& \qquad \times \left( \frac{2 (|m| + s)}{\sqrt{s^2 + 2 |m| s}} - \frac{ I_{m-1} \left( \sqrt{s^2 + 2 |m| s} \right) + I_{m+1} \left( \sqrt{s^2 + 2 |m| s} \right) }{ I_m \left( \sqrt{s^2 + 2 |m| s} \right)} \right)^2 \\
& \qquad + \frac{(1 - p)^2 \exp(-|m| - s)}{\left( s^2 + 2 |m| s \right) \left( p \exp(|m| + s) + (1 - p) I_0 \left( \sqrt{s^2 + 2 |m| s} \right) \right)} \\
& \qquad \times \left( \sqrt{s^2 + 2 |m| s} I_0 \left( \sqrt{s^2 + 2 |m| s} \right) - (|m| + s) I_1 \left( \sqrt{s^2 + 2 |m| s} \right) \right)^2 \\
\mathcal{I}_{s, p} (m, s, p) &= \frac{(p - 1) \left( 1 - \exp(-|m| - s) I_0 \left( \sqrt{s^2 + 2 |m| s} \right) \right)}{\sqrt{s^2 + 2 |m| s} \left( p \exp(|m| + s) + (1 - p) I_0 \left( \sqrt{s^2 + 2 |m| s} \right) \right)} \\
& \qquad \times \left( \sqrt{s^2 + 2 |m| s} I_0 \left( \sqrt{s^2 + 2 |m| s} \right) - (|m| + s) I_1 \left( \sqrt{s^2 + 2 |m| s} \right) \right) \\
\mathcal{I}_{p, p} (m, s, p) &= \frac{\exp(|m| + s) - I_0 \left( \sqrt{s^2 + 2 |m| s} \right)}{(1 - p) \left( p \exp(|m| + s) + (1 - p) I_0 \left( \sqrt{s^2 + 2 |m| s} \right) \right)} \\
\end{aligned}
$$

### Mean-Variance Parametrization

#### Parameters

- Mean parameter $m \in \mathbb{R}$
- Variance parameter $s \in (|m|, \infty)$
- Inflation parameter $p \in (0, 1)$

#### Probability Mass Function

$$
\begin{aligned}
\mathrm{P} [Y = y | m, s, p] &= \begin{cases}
p + (1 - p) \exp(-s) I_0 \left( \sqrt{s^2 - m^2} \right) & \text{ for } y = 0 \\
(1 - p) \exp(-s) \left( \frac{s + m}{s - m} \right)^{\frac{y}{2}} I_y \left( \sqrt{s^2 - m^2} \right) & \text{ for } y \neq 0 \\
\end{cases} \\
\end{aligned}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= (1 - p) m \\
\mathrm{var}[Y] &= (1 - p) \left( s + p m^2 \right) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s, p) &= \begin{cases}
\frac{m (p - 1) I_{1} \left( \sqrt{s^2 - m^2} \right)}{\sqrt{s^2 - m^2} \left( p \exp(s) + (1 - p) I_{0} \left( \sqrt{s^2 - m^2} \right) \right)} & \text{ for } y = 0 \\
\frac{s y}{s^2 - m^2} - \frac{m}{2 \sqrt{s^2 - m^2}} \frac{ I_{y-1} \left( \sqrt{s^2 - m^2} \right) + I_{y+1} \left( \sqrt{s^2 - m^2} \right) }{ I_y \left( \sqrt{s^2 - m^2} \right) } & \text{ for } y \neq 0 \\
\end{cases} \\
\nabla_{s} (y; m, s, p) &= \begin{cases}
\frac{ (p - 1) \left( \sqrt{s^2 - m^2} I_{0} \left( \sqrt{s^2 - m^2} \right) - s I_{1} \left( \sqrt{s^2 - m^2} \right) \right) }{\sqrt{s^2 - m^2} \left( p \exp(s) + (1 - p) I_{0} \left( \sqrt{s^2 - m^2} \right) \right)} & \text{ for } y = 0 \\
-\frac{m y}{s^2 - m^2} + \frac{s}{2 \sqrt{s^2 - m^2}} \frac{ I_{y-1} \left( \sqrt{s^2 - m^2} \right) + I_{y+1} \left( \sqrt{s^2 - m^2} \right) }{ I_y \left( \sqrt{s^2 - m^2} \right) } - 1 & \text{ for } y \neq 0 \\
\end{cases} \\
\nabla_{p} (y; m, s, p) &= \begin{cases}
\frac{\exp(s) - I_{0} \left( \sqrt{s^2 - m^2} \right)}{p \exp(s) + (1 - p) I_{0} \left( \sqrt{s^2 - m^2} \right)} & \text{ for } y = 0 \\
\frac{1}{p - 1} & \text{ for } y \neq 0 \\
\end{cases} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s, p) &\approx \frac{m^2 (1 - p) \left( 1 - \exp(-s) I_{0} \left( \sqrt{s^2 - m^2} \right) \right)}{4 \left( s^2 - m^2 \right)} \\
& \qquad \times \left( \frac{2 s}{\sqrt{s^2 - m^2}} - \frac{ I_{m-1} \left( \sqrt{s^2 - m^2} \right) + I_{m+1} \left( \sqrt{s^2 - m^2} \right) }{ I_m \left( \sqrt{s^2 - m^2} \right) } \right)^2 \\
& \qquad + \frac{m^2 (1 - p)^2 \exp(-s) I_{1} \left( \sqrt{s^2 - m^2} \right)^2}{\left( s^2 - m^2 \right) \left( p \exp(s) + (1 - p) I_0 \left( \sqrt{s^2 - m^2} \right) \right)} \\
\mathcal{I}_{m, s} (m, s, p) &\approx \frac{m s (p - 1) \left( 1 - \exp(-s) I_{0} \left( \sqrt{s^2 - m^2} \right) \right) }{4 \left( s^2 - m^2 \right)} \\
& \qquad \times \left( \frac{2 s}{\sqrt{s^2 - m^2}} - \frac{ I_{m-1} \left( \sqrt{s^2 - m^2} \right) + I_{m+1} \left( \sqrt{s^2 - m^2} \right) }{ I_m \left( \sqrt{s^2 - m^2} \right) } \right)^2 \\
& \qquad + \frac{m (1 - p)^2 \exp(-s) I_{1} \left( \sqrt{s^2 - m^2} \right)}{\left( s^2 - m^2 \right) \left( p \exp(s) + (1 - p) I_0 \left( \sqrt{s^2 - m^2} \right) \right)} \\
& \qquad \times \left( \sqrt{s^2 - m^2} I_0 \left( \sqrt{s^2 - m^2} \right) - s I_1 \left( \sqrt{s^2 - m^2} \right) \right) \\
\mathcal{I}_{m, p} (m, s, p) &= \frac{m (p - 1) \left( 1 - \exp(-s) I_0 \left( \sqrt{s^2 - m^2} \right) \right) I_1 \left( \sqrt{s^2 - m^2} \right)}{\sqrt{s^2 - m^2} \left( p \exp(s) + (1 - p) I_0 \left( \sqrt{s^2 - m^2} \right) \right)} \\
\mathcal{I}_{s, s} (m, s, p) &\approx \frac{s^2 (1 - p) \left( 1 - \exp(-s) I_{0} \left( \sqrt{s^2 - m^2} \right) \right)}{4 \left( s^2 - m^2 \right)} \\
& \qquad \times \left( \frac{2 s}{\sqrt{s^2 - m^2}} - \frac{ I_{m-1} \left( \sqrt{s^2 - m^2} \right) + I_{m+1} \left( \sqrt{s^2 - m^2} \right) }{ I_m \left( \sqrt{s^2 - m^2} \right) } \right)^2 \\
& \qquad + \frac{(1 - p)^2 \exp(-s) \left( \sqrt{s^2 - m^2} I_0 \left( \sqrt{s^2 - m^2} \right) - s I_1 \left( \sqrt{s^2 - m^2} \right) \right)^2}{\left( s^2 - m^2 \right) \left( p \exp(s) + (1 - p) I_0 \left( \sqrt{s^2 - m^2} \right) \right)} \\
\mathcal{I}_{s, p} (m, s, p) &= \frac{(p - 1) \left( 1 - \exp(-s) I_0 \left( \sqrt{s^2 - m^2} \right) \right) }{\sqrt{s^2 - m^2} \left( p \exp(s) + (1 - p) I_0 \left( \sqrt{s^2 - m^2} \right) \right)} \\
& \qquad \times \left( \sqrt{s^2 - m^2} I_0 \left( \sqrt{s^2 - m^2} \right) - s I_1 \left( \sqrt{s^2 - m^2} \right) \right) \\
\mathcal{I}_{p, p} (m, s, p) &= \frac{\exp(s) - I_0 \left( \sqrt{s^2 - m^2} \right)}{(1 - p) \left( p \exp(s) + (1 - p) I_0 \left( \sqrt{s^2 - m^2} \right)  \right) } \\
\end{aligned}
$$

### Note

- The computation of the Fisher information for the first two parameters is quite intricate and we resort to an approximation by replacing $y$ with its expected value combined with the zero value.

### Further Reading

- Karlis, D. and Ntzoufras, I. (2009). Bayesian Modelling of Football Outcomes: Using the Skellam’s Distribution for the Goal Difference. *IMA Journal of Management Mathematics*, **20**(2), 133–145. doi: [10.1093/imaman/dpn026](https://doi.org/10.1093/imaman/dpn026).

# Circular Data

## von Mises Distribution

### Mean-Concentration Parametrization

#### Parameters

- Mean parameter $m \in \mathbb{R}$
- Concentration parameter $v \in (0, \infty)$

#### Density Function

$$
f(y | m, v) = \frac{1}{2 \pi I_0(v)} \exp \left( v \cos(y - m) \right)
$$

#### Circular Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m \\
\mathrm{var}[Y] &= 1 - \frac{I_1(v)}{I_0(v)} \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, v) &= v \sin(y - m) \\
\nabla_{v} (y; m, v) &= \cos(y - m) - \frac{I_1(v)}{I_0(v)} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, v) &= v \frac{I_1(v)}{I_0(v)} \\
\mathcal{I}_{m, v} (m, v) &= 0 \\
\mathcal{I}_{v, v} (m, v) &= \frac{1}{2} - \left( \frac{I_1(v)}{I_0(v)} \right)^2 + \frac{I_2(v)}{2 I_0(v)} \\
\end{aligned}
$$

### Further Reading

- Harvey, A., Hurn, S., and Thiele, S. (2019). Modeling Directional (Circular) Time Series. *Cambridge Working Papers in Economics*, CWPE 1971. doi: [10.17863/cam.43915](https://doi.org/10.17863/cam.43915).

# Interval Data

## Beta Distribution

### Concentration Parametrization

#### Parameters

- First concentration parameter $a_1 \in (0, \infty)$
- Second concentration parameter $a_2 \in (0, \infty)$

#### Density Function

$$
f(y | a_1, a_2) = \frac{1}{B(a_1, a_2)} y^{a_1 - 1} (1 - y)^{a_2 - 1}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= \frac{a_1}{a_1 + a_2} \\
\mathrm{var}[Y] &= \frac{a_1 a_2}{(a_1 + a_2)^2 (a_1 + a_2 + 1)} \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{a} (y; a_1, a_2) &= \psi_0(a_1 + a_2) - \psi_0(a_1) + \ln(y) \\
\nabla_{b} (y; a_1, a_2) &= \psi_0(a_1 + a_2) - \psi_0(a_2) + \ln(1 - y) \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{a_1, a_1} (a_1, a_2) &= \psi_1(a_1) - \psi_1(a_1 + a_2) \\
\mathcal{I}_{a_1, a_2} (a_1, a_2) &= -\psi_1(a_1 + a_2) \\
\mathcal{I}_{a_2, a_2} (a_1, a_2) &= \psi_1(a_2) - \psi_1(a_1 + a_2) \\
\end{aligned}
$$

### Mean-Size Parametrization

#### Parameters

- Mean parameter $m \in (0, 1)$
- Size parameter $v \in (0, \infty)$

#### Density Function

$$
f(y | m, v) = \frac{1}{B(m v, (1 - m) v)} y^{m v - 1} (1 - y)^{(1 - m) v - 1}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m \\
\mathrm{var}[Y] &= \frac{m (1 - m)}{v + 1} \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, v) &= \frac{v}{1 - m} (\psi_0(v) - \psi_0(m v) + \ln(y)) \\
\nabla_{v} (y; m, v) &= \psi_0(v) - \psi_0(v - m v) + \ln(1 - y) \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, v) &= \frac{v^2}{(1 - m)^2} (\psi_1(m v) - \psi_1(v)) \\
\mathcal{I}_{m, v} (m, v) &= \frac{v}{m - 1} \psi_1(v) \\
\mathcal{I}_{v, v} (m, v) &= \psi_1(v - m v) - \psi_1(v) \\
\end{aligned}
$$

### Mean-Variance Parametrization

#### Parameters

- Mean parameter $m \in (0, 1)$
- Variance parameter $s \in (0, m (1 - m))$

#### Density Function

$$
f(y | m, s) = \frac{1}{B \left( m \left( \frac{m - m^2}{s} - 1 \right), (1 - m) \left( \frac{m - m^2}{s} - 1 \right) \right)} y^{m \left( \frac{m - m^2}{s} - 1 \right) - 1} (1 - y)^{(1 - m) \left( \frac{m - m^2}{s} - 1 \right) - 1}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m \\
\mathrm{var}[Y] &= s \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s) &= \frac{m^2 - m + s}{(m - 1) s} \left( \psi_0 \left( \frac{m - m^2}{s} - 1 \right) - \psi_0 \left( m \left( \frac{m - m^2}{s} - 1 \right) \right) + \ln(y) \right) \\
\nabla_{s} (y; m, s) &= \frac{s^2 (3 m^2 - 2 m + s)}{m (m - 1) (m^2 - m + s)}  \left( \psi_0 \left( \frac{m - m^2}{s} - 1 \right) - \psi_0 \left( (1 - m) \left( \frac{m - m^2}{s} - 1 \right) \right) + \ln(1 - y) \right) \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s) &= \frac{(m^2 - m + s)^2}{(m - 1)^2 s^2} \left( \psi_1 \left( m \left( \frac{m - m^2}{s} - 1 \right) \right)  - \psi_1 \left( \frac{m - m^2}{s} - 1 \right) \right) \\
\mathcal{I}_{m, s} (m, s) &= \frac{s (2 m - 3 m^2 - s)}{m (m^2 - 2 m + 1)} \psi_1 \left( \frac{m - m^2}{s} - 1 \right) \\
\mathcal{I}_{s, s} (m, s) &= \frac{s^2 (3 m^2 - 2 m + s)^2}{(m - 1)^4 m^2} \left( \psi_1 \left( (1 - m) \left( \frac{m - m^2}{s} - 1 \right) \right)  - \psi_1 \left( \frac{m - m^2}{s} - 1 \right) \right) \\
\end{aligned}
$$

## Kumaraswamy Distribution

### Concentration Parametrization

#### Parameters

- First concentration parameter $a \in (0, \infty)$
- Second concentration parameter $b \in (0, \infty)$

#### Density Function

$$
f(y | a, b) = a b y^{a - 1} \left(1 - y^a \right)^{b - 1}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= b B \left(1 + \frac{1}{a}, b \right) \\
\mathrm{var}[Y] &= b B \left(1 + \frac{2}{a}, b \right) - b^2 B \left(1 + \frac{1}{a}, b \right)^2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{a} (y; a, b) &= \frac{(b - 1) \ln(y)}{y^a - 1} + b \ln(y) + \frac{1}{a} \\
\nabla_{b} (y; a, b) &= \ln \left( 1 - y^a \right) + \frac{1}{b} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{a, a} (a, b) &= \frac{1}{a^2} + \frac{b}{a^2 (b - 2)} \left( \left(\psi_0(b) - \psi_0(2) \right)^2 - \left( \psi_1(b) - \psi_1(2) \right) \right) \\
\mathcal{I}_{a, b} (a, b) &= - \frac{\psi_0(b + 1) - \psi_0(2)}{a (b - 1)} \\
\mathcal{I}_{b, b} (a, b) &= \frac{1}{b^2} \\
\end{aligned}
$$

### Further Reading

- Jones, M. C. (2009). Kumaraswamy's Distribution: A Beta-Type Distribution with Some Tractability Advantages. *Statistical Methodology*, **6**(1), 70–81. doi: [10.1016/j.stamet.2008.04.001](https://doi.org/10.1016/j.stamet.2008.04.001).

## Logit-Normal Distribution

### Logit-Mean-Variance Parametrization

#### Parameters

- Logit-mean parameter $m \in \mathbb{R}$
- Logit-variance parameter $s \in (0, \infty)$

#### Density Function

$$
f(y | m, s) = \frac{1}{y (1 - y)}\frac{1}{\sqrt{2 \pi s}} \exp \left( - \frac{1}{2 s} \left( \ln \left( \frac{y}{1-y} \right) - m \right)^2 \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &\approx \frac{1}{K - 1} \sum_{k = 1}^{K - 1} \frac{1}{1 + \exp( - \Phi^{-1}_{m,s} (k / K)} \\
\mathrm{var}[Y] &\approx \frac{1}{K - 1} \sum_{k = 1}^{K - 1} \left( \frac{1}{1 + \exp( - \Phi^{-1}_{m,s} (k / K)} \right)^2 - \left( \frac{1}{K - 1} \sum_{k = 1}^{K - 1} \frac{1}{1 + \exp( - \Phi^{-1}_{m,s} (k / K)} \right)^2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s) &= \frac{1}{s} \left( \ln \left( \frac{y}{1-y} \right) - m \right) \\
\nabla_{s} (y; m, s) &= \frac{1}{2 s^2} \left( \ln \left( \frac{y}{1-y} \right) - m \right)^2 - \frac{1}{2s} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s) &= \frac{1}{s} \\
\mathcal{I}_{m, s} (m, s) &= 0 \\
\mathcal{I}_{s, s} (m, s) &= \frac{1}{2 s^2} \\
\end{aligned}
$$

### Note

- The mean and variance have no analytic solution. We use the quasi Monte Carlo approximation with $K=1000$. 

# Compositional Data

## Dirichlet Distribution

### Concentration Parametrization

#### Parameters

- Concentration parameters $a_i \in (0, \infty)$, $i = 1,\ldots,n$

#### Vector Notation

- Concentration vector $\boldsymbol{a}$ of length $n$

#### Density Function

$$
f(\boldsymbol{y} | \boldsymbol{a}) = \frac{1}{B(\boldsymbol{a})} \prod_{i=1}^n y_i^{a_i - 1}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[\boldsymbol{Y}] &= \frac{1}{\sum_{i=1}^n a_i} \boldsymbol{a} \\
\mathrm{var}[\boldsymbol{Y}] &= \frac{1}{1 + \sum_{i=1}^n a_i} \left( \frac{1}{\sum_{i=1}^n a_i} \mathrm{diag}(\boldsymbol{a}) - \frac{1}{\left( \sum_{i=1}^n a_i \right)^2} \boldsymbol{a} \boldsymbol{a}^\intercal \right) \\
\end{aligned}
$$

#### Score

$$
\nabla_{\boldsymbol{a}} (\boldsymbol{y}; \boldsymbol{a}) = \ln(\boldsymbol{y}) - \psi_0 (\boldsymbol{a}) + \psi_0 \left( \sum_{i=1}^n a_i \right) \\
$$

#### Fisher Information

$$
\mathcal{I}_{\boldsymbol{a}, \boldsymbol{a}} (\boldsymbol{a}) = \mathrm{diag} \left( \psi_1 \left( \boldsymbol{a} \right) \right) - \psi_1 \left( \sum_{i=1}^n a_i \right)  \\
$$

### Further Reading

- Calvori, F., Cipollini, F., and Gallo, G. M. (2013). Go with the Flow: A GAS Model For Predicting Intra-Daily Volume Shares. *SSRN*, 2363483. doi: [10.2139/ssrn.2363483](https://doi.org/10.2139/ssrn.2363483).

# Duration Data

## Birnbaum–Saunders Distribution

### Scale Parametrization

#### Parameters

- Scale parameter $s \in (0, \infty)$
- Shape parameter $a \in (0, \infty)$

#### Density Function

$$
f(y | s, a) = \frac{\sqrt{\frac{s}{y}} \left( 1 + \frac{s}{y} \right)}{2 a s \sqrt{2 \pi}} \exp \left( \frac{2 - \frac{y}{s} - \frac{s}{y}}{2 a^2} \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= s \left( 1 + \frac{a^2}{2} \right) \\
\mathrm{var}[Y] &= s^2 a^2 \left( 1 + \frac{5 a^2}{4} \right) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{s} (y; s, a) &= \frac{y}{2 a^2 s^2} - \frac{1}{2 a^2 y} + \frac{1}{s + y} - \frac{1}{2 s} \\
\nabla_{a} (y; s, a) &= \frac{y}{a^3 s} + \frac{s}{a^3 y} - \frac{2 + a^2}{a^3} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{s, s} (s, a) &= \frac{1}{a^2 s^2} \left( 1 + \frac{a}{\sqrt{2 \pi}} \left( a \sqrt{\frac{\pi}{2}} - \pi \exp \left( \frac{2}{a^2} \right) \left(1 - \Phi \left( \frac{2}{a} \right) \right) \right) \right) \\
\mathcal{I}_{s, a} (s, a) &= 0 \\
\mathcal{I}_{a, a} (s, a) &= \frac{2}{a^2} \\
\end{aligned}
$$

### Further Reading

- Lemonte, A. J. (2021). A Note on the Fisher Information Matrix of the Birnbaum–Saunders Distribution. *Journal of Statistical Theory and Applications*, **15**(2), 196–205. doi: [10.2991/jsta.2016.15.2.9](https://doi.org/10.2991/jsta.2016.15.2.9).

## Burr Distribution

### Scale Parametrization

#### Parameters

- Scale parameter $s \in (0, \infty)$
- First shape parameter $a \in (0, \infty)$
- Second shape parameter $b \in (0, \infty)$

#### Density Function

$$
f(y | s, a, b) = \frac{a b}{s} \left( \frac{y}{s} \right)^{a - 1} \left( 1 + \left( \frac{y}{s} \right)^a \right)^{-b - 1}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= s b B \left( b - \frac{1}{a}, 1 + \frac{1}{a} \right), & \quad \text{for } a &> 1 \\
\mathrm{var}[Y] &= s^2 b B \left( b - \frac{2}{a}, 1 + \frac{2}{a} \right) - s^2 b^2 B \left( b - \frac{1}{a}, 1 + \frac{1}{a} \right)^2, & \quad \text{for } a &> 2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{s} (y; s, a, b) &= \frac{a}{s} \left(b \left (\frac{y}{s} \right)^a - 1 \right) \left( \left( \frac{y}{s} \right)^a + 1 \right)^{-1} \\
\nabla_{a} (y; s, a, b) &= \frac{1}{a} - \left( b \left( \frac{y}{s} \right)^a - 1 \right) \ln \left( \frac{y}{s} \right) \left( \left( \frac{y}{s} \right)^a + 1 \right)^{-1} \\
\nabla_{b} (y; s, a, b) &= \frac{1}{b} - \ln \left( \left( \frac{y}{s} \right)^a + 1 \right) \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{s, s} (s, a, b) &= \frac{a^2 b}{s^2 (b + 2)} \\
\mathcal{I}_{s, a} (s, a, b) &= - \frac{b ( 1 - \gamma - \psi_0(b + 1))}{s (b + 2)} \\
\mathcal{I}_{s, b} (s, a, b) &= - \frac{a}{s (b + 1)} \\
\mathcal{I}_{a, a} (s, a, b) &= \frac{1}{\alpha^2} \left( 1 + \frac{b}{b + 2} \left( \frac{\pi^2}{6} + \gamma^2 - 2 \gamma + 2 (\gamma - 1) \psi_0(b + 1) + \psi_0(b + 1)^2 + \psi_1(b + 1) \right) \right) \\
\mathcal{I}_{a, b} (s, a, b) &= \frac{1 - \gamma - \psi_0(b)}{a (b + 1)} \\
\mathcal{I}_{b, b} (s, a, b) &= \frac{1}{b^2} \\
\end{aligned}
$$

### Further Reading

- Watkins, A. J. (1997). A Note on Expected Fisher Information for the Burr XII Distribution. *Microelectronics Reliability*, **37**(12), 1849–1852. doi: [10.1016/s0026-2714(97)00030-9](https://doi.org/10.1016/s0026-2714(97)00030-9).

## Exponential Distribution

### Rate Parametrization

#### Parameter

- Rate parameter $r \in (0, \infty)$

#### Density Function

$$
f(y | r) = r \exp \left( -r y \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= \frac{1}{r} \\
\mathrm{var}[Y] &= \frac{1}{r^2} \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{r} (y; r) &= \frac{1}{r} - y \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{r, r} (r) &= \frac{1}{r^2} \\
\end{aligned}
$$

### Scale Parametrization

#### Parameter

- Scale parameter $s \in (0, \infty)$

#### Density Function

$$
f(y | s) =  \frac{1}{s} \exp \left( - \frac{y}{s} \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= s \\
\mathrm{var}[Y] &= s^2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{s} (y; s) &= \frac{y - s}{s^2} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{s, s} (s) &= \frac{1}{s^2} \\
\end{aligned}
$$

### Further Reading

- Tomanová, P. and Holý, V. (2021). Clustering of Arrivals in Queueing Systems: Autoregressive Conditional Duration Approach. *Central European Journal of Operations Research*, **29**(3), 859–874. doi: [10.1007/s10100-021-00744-7](https://doi.org/10.1007/s10100-021-00744-7).

## Exponential-Logarithmic Distribution

### Rate Parametrization

#### Parameters

- Rate parameter $r \in (0, \infty)$
- Shape parameter $p \in (0, 1)$

#### Density Function

$$
f(y | r, p) = \frac{r}{- \ln(p)} \frac{(1 - p) \exp(-r y)}{1 - (1 - p) \exp(-r y)}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= - \frac{\mathrm{Li}_2(1 - p)}{r \ln(p)} \\
\mathrm{var}[Y] &= - 2 \frac{\mathrm{Li}_3(1 - p)}{r^2 \ln(p)} - \left( \frac{\mathrm{Li}_2(1 - p)}{r \ln(p)} \right)^2
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{r} (y; r, p) &= \frac{1}{r} - y - \frac{y (1 - p) \exp(-r y)}{1 - (1 - p) \exp(-r y)} \\
\nabla_{p} (y; r, p) &= -\frac{1}{p log(p)} - \frac{1}{1 - p} - \frac{\exp(-r y)}{1 - (1 - p) \exp(-r y)} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{r, r} (r, p) &= -\frac{\mathrm{Li}_2(1 - p)}{r^2 \ln(p)} \\
\mathcal{I}_{r, p} (r, p) &= \frac{1 - p + p \ln(p)}{2 r p (1 - p) \ln(p)} \\
\mathcal{I}_{p, p} (r, p) &= \frac{1}{(1 - p)^2} - \frac{\ln(p) + 1}{(p \ln(p))^2} + \frac{1 - 4 p + 3 p^2 - 2 p^2 \ln(p)}{2 p^2 (1 - p)^2 \ln(p)} \\
\end{aligned}
$$

### Further Reading

- Tahmasbi, R. and Rezaei, S. (2008). A Two-Parameter Lifetime Distribution with Decreasing Failure Rate. *Computational Statistics and Data Analysis*, **52**(8), 3889–3901 doi: [10.1016/j.csda.2007.12.002](https://doi.org/10.1016/j.csda.2007.12.002).

## Fisk Distribution

### Scale Parametrization

#### Parameters

- Scale parameter $s \in (0, \infty)$
- Shape parameter $a \in (0, \infty)$

#### Density Function

$$
f(y | s, a) = \frac{a}{s} \left( \frac{y}{s} \right)^{a - 1} \left( 1 + \left ( \frac{y}{s} \right)^a \right)^{-2}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= s \frac{\pi / a}{\sin(\pi / a)}, & \quad \text{for } a &> 1 \\
\mathrm{var}[Y] &= s^2  \left( \frac{2 \pi / a}{\sin(2 \pi / a)} - \frac{\pi^2 / a^2}{\sin(\pi / a)^2} \right), & \quad \text{for } a &> 2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{s} (y; s, a) &= \frac{a}{s} \left( \left( \frac{y}{s} \right)^a - 1 \right) \left( \left( \frac{y}{s} \right)^a + 1 \right)^{-1} \\
\nabla_{a} (y; s, a) &= \frac{1}{a} - \left( \left( \frac{y}{s} \right)^a - 1 \right) \ln \left( \frac{y}{s} \right) \left( \left( \frac{y}{s} \right)^a + 1 \right)^{-1} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{s, s} (s, a) &= \frac{a^2}{3 s^2} \\
\mathcal{I}_{s, a} (s, a) &= 0 \\
\mathcal{I}_{a, a} (s, a) &= \frac{\pi^2 + 3}{9 a^2} \\
\end{aligned}
$$

## Gamma Distribution

### Rate Parametrization

#### Parameters

- Rate parameter $r \in (0, \infty)$
- Shape parameter $a \in (0, \infty)$

#### Density Function

$$
f(y | r, a) =  \frac{r}{\Gamma(a)} (r y)^{a - 1} \exp \left( -r y \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= \frac{a}{r} \\
\mathrm{var}[Y] &= \frac{a}{r^2} \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{r} (y; r, a) &= \frac{a - r y}{r} \\
\nabla_{a} (y; r, a) &= \ln(r y) - \psi_0(a) \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{r, r} (r, a) &= \frac{a}{r^2} \\
\mathcal{I}_{r, a} (r, a) &= - \frac{1}{r} \\
\mathcal{I}_{a, a} (r, a) &= \psi_1(a) \\
\end{aligned}
$$

### Scale Parametrization

#### Parameters

- Scale parameter $s \in (0, \infty)$
- Shape parameter $a \in (0, \infty)$

#### Density Function

$$
f(y | s, a) =  \frac{1}{\Gamma(a)} \frac{1}{s} \left( \frac{y}{s} \right)^{a - 1} \exp \left( - \frac{y}{s} \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= a s \\
\mathrm{var}[Y] &= a s^2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{s} (y; s, a) &= \frac{y - a s}{s^2} \\
\nabla_{a} (y; s, a) &= \ln \left( \frac{y}{s} \right) - \psi_0(a) \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{s, s} (s, a) &= \frac{a}{s^2} \\
\mathcal{I}_{s, a} (s, a) &= \frac{1}{s} \\
\mathcal{I}_{a, a} (s, a) &= \psi_1(a) \\
\end{aligned}
$$

### Further Reading

- Tomanová, P. and Holý, V. (2021). Clustering of Arrivals in Queueing Systems: Autoregressive Conditional Duration Approach. *Central European Journal of Operations Research*, **29**(3), 859–874. doi: [10.1007/s10100-021-00744-7](https://doi.org/10.1007/s10100-021-00744-7).

## Generalized Gamma Distribution

### Rate Parametrization

#### Parameters

- Rate parameter $r \in (0, \infty)$
- First shape parameter $a \in (0, \infty)$
- Second shape parameter $b \in (0, \infty)$

#### Density Function

$$
f(y | r, a, b) =  \frac{r b}{\Gamma(a)} (r y)^{a b - 1} \exp \left( -(r y)^b \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= \frac{1}{r} \frac{\Gamma \left(a + b^{-1} \right)}{\Gamma \left( a \right) } \\
\mathrm{var}[Y] &= \frac{1}{r^2} \left( \frac{\Gamma \left(a + 2 b^{-1} \right)}{\Gamma \left( a \right) } -  \left( \frac{\Gamma \left(a + b^{-1} \right)}{\Gamma \left( a \right) } \right)^2 \right) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{r} (y; r, a, b) &= \frac{b}{r} \left( a - (r y)^b \right) \\
\nabla_{a} (y; r, a, b) &= b \ln(r y) - \psi_0(a) \\
\nabla_{b} (y; r, a, b) &= \left( a - (r y)^b \right) \ln (r y) + \frac{1}{b} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{r, r} (r, a, b) &= \frac{a b^2}{r^2} \\
\mathcal{I}_{r, a} (r, a, b) &= - \frac{b}{r} \\
\mathcal{I}_{r, b} (r, a, b) &= \frac{a \psi_0(a) + 1}{r} \\
\mathcal{I}_{a, a} (r, a, b) &= \psi_1(a) \\
\mathcal{I}_{a, b} (r, a, b) &= - \frac{\psi_0(a)}{b} \\
\mathcal{I}_{b, b} (r, a, b) &= \frac{a \psi_0(a)^2 + 2 \psi_0(a) + a \psi_1(a) + 1}{b^2} \\
\end{aligned}
$$

### Scale Parametrization

#### Parameters

- Scale parameter $s \in (0, \infty)$
- First shape parameter $a \in (0, \infty)$
- Second shape parameter $b \in (0, \infty)$

#### Density Function

$$
f(y | s, a, b) =  \frac{1}{\Gamma(a)} \frac{b}{s} \left( \frac{y}{s} \right)^{a b - 1} \exp \left( - \left( \frac{y}{s} \right)^b \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= s \frac{\Gamma \left(a + b^{-1} \right)}{\Gamma \left( a \right) } \\
\mathrm{var}[Y] &= s^2 \left( \frac{\Gamma \left(a + 2 b^{-1} \right)}{\Gamma \left( a \right) } -  \left( \frac{\Gamma \left(a + b^{-1} \right)}{\Gamma \left( a \right) } \right)^2 \right) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{s} (y; s, a, b) &= \frac{b}{s} \left( \left( \frac{y}{s} \right)^b - a \right) \\
\nabla_{a} (y; s, a, b) &= b \ln \left( \frac{y}{s} \right) - \psi_0(a) \\
\nabla_{b} (y; s, a, b) &= \left( a - \left( \frac{y}{s} \right)^b \right) \ln \left( \frac{y}{s} \right) + \frac{1}{b} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{s, s} (s, a, b) &= \frac{a b^2}{s^2} \\
\mathcal{I}_{s, a} (s, a, b) &= \frac{b}{s} \\
\mathcal{I}_{s, b} (s, a, b) &= - \frac{a \psi_0(a) + 1}{s} \\
\mathcal{I}_{a, a} (s, a, b) &= \psi_1(a) \\
\mathcal{I}_{a, b} (s, a, b) &= - \frac{\psi_0(a)}{b} \\
\mathcal{I}_{b, b} (s, a, b) &= \frac{a \psi_0(a)^2 + 2 \psi_0(a) + a \psi_1(a) + 1}{b^2} \\
\end{aligned}
$$

### Further Reading

- Park, T. R. (2014). Derivation of the Fisher Information Matrix for 4-Parameter Generalized Gamma Distribution Using Mathematica. *Journal of the Chosun Natural Science*, **7**(2), 138–144. doi: [10.13160/ricns.2014.7.2.138](https://doi.org/10.13160/ricns.2014.7.2.138).

- Stacy, E. W. (1962). A Generalization of the Gamma Distribution. *The Annals of Mathematical Statistics*, **33**(3), 1187–1192. doi: [10.1214/aoms/1177704481](https://doi.org/10.1214/aoms/1177704481).

- Tomanová, P. and Holý, V. (2021). Clustering of Arrivals in Queueing Systems: Autoregressive Conditional Duration Approach. *Central European Journal of Operations Research*, **29**(3), 859–874. doi: [10.1007/s10100-021-00744-7](https://doi.org/10.1007/s10100-021-00744-7).

## Log-Normal Distribution

### Log-Mean-Variance Parametrization

#### Parameters

- Log-mean parameter $m \in \mathbb{R}$
- Log-variance parameter $s \in (0, \infty)$

#### Density Function

$$
f(y | m, s) = \frac{1}{y}\frac{1}{\sqrt{2 \pi s}} \exp \left( - \frac{\left( \ln(y) - m \right)^2}{2 s} \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= \exp \left( m + \frac{s}{2} \right) \\
\mathrm{var}[Y] &= \left( \exp(s) - 1 \right) \exp \left( 2 m + s \right) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s) &= \frac{\ln(y) - m}{s}  \\
\nabla_{s} (y; m, s) &= \frac{\left( \ln(y) - m \right)^2}{2 s^2} - \frac{1}{2s} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s) &= \frac{1}{s} \\
\mathcal{I}_{m, s} (m, s) &= 0 \\
\mathcal{I}_{s, s} (m, s) &= \frac{1}{2 s^2} \\
\end{aligned}
$$

## Lomax Distribution

### Scale Parametrization

#### Parameters

- Scale parameter $s \in (0, \infty)$
- Shape parameter $b \in (0, \infty)$

#### Density Function

$$
f(y | s, b) = \frac{b}{s} \left( 1 + \frac{y}{s} \right)^{-b - 1}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= \frac{s}{b - 1}, & \quad \text{for } a &> 1 \\
\mathrm{var}[Y] &= \frac{s^2 b}{(b - 1)^2 (b - 2)}, & \quad \text{for } a &> 2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{s} (y; s, b) &= \frac{1}{s} \left( b \frac{y}{s} - 1 \right) \left( \frac{y}{s} + 1 \right)^{-1} \\
\nabla_{b} (y; s, b) &= \frac{1}{b} - \ln \left( \frac{y}{s} + 1 \right) \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{s, s} (s, b) &= \frac{b}{s^2 (b + 2)} \\
\mathcal{I}_{s, b} (s, b) &= - \frac{1}{s (b + 1)} \\
\mathcal{I}_{b, b} (s, b) &= \frac{1}{b^2} \\
\end{aligned}
$$

## Rayleigh Distribution

### Scale Parametrization

#### Parameter

- Scale parameter $s \in (0, \infty)$

#### Density Function

$$
f(y | s) =  \frac{y}{s^2} \exp \left(- \frac{y^2}{2 s^2} \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= s \sqrt{\frac{\pi}{2}} \\
\mathrm{var}[Y] &= s^2 \frac{4 - \pi}{2}\\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{s} (y; s) &= \frac{y^2 - 2 s^2}{s^3} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{s, s} (s) &= \frac{4}{s^2} \\
\end{aligned}
$$

## Weibull Distribution

### Rate Parametrization

#### Parameters

- Rate parameter $r \in (0, \infty)$
- Shape parameter $b \in (0, \infty)$

#### Density Function

$$
f(y | r, b) =  r b (r y)^{b - 1} \exp \left( -(r y)^b \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= \frac{1}{r} \Gamma \left(1 + b^{-1} \right) \\
\mathrm{var}[Y] &= \frac{1}{r^2} \left( \Gamma \left(1 + 2 b^{-1} \right) -  \Gamma \left(1 + b^{-1} \right)^2 \right) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{r} (y; r, b) &= \frac{b}{r} \left( 1 - (r y)^b \right) \\
\nabla_{b} (y; r, b) &= \left( 1 - (r y)^b \right) \ln (r y) + \frac{1}{b} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{r, r} (r, b) &= \frac{b^2}{r^2} \\
\mathcal{I}_{r, b} (r, b) &= \frac{\psi_0(1) + 1}{r} \\
\mathcal{I}_{b, b} (r, b) &= \frac{\psi_0(1)^2 + 2 \psi_0(1) + \psi_1(1) + 1}{b^2} \\
\end{aligned}
$$

### Scale Parametrization

#### Parameters

- Scale parameter $s \in (0, \infty)$
- Shape parameter $b \in (0, \infty)$

#### Density Function

$$
f(y | s, b) =  \frac{b}{s} \left( \frac{y}{s} \right)^{b - 1} \exp \left( - \left( \frac{y}{s} \right)^b \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= s \Gamma \left(1 + b^{-1} \right) \\
\mathrm{var}[Y] &= s^2 \left( \Gamma \left(1 + 2 b^{-1} \right) -  \Gamma \left(1 + b^{-1} \right)^2 \right) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{s} (y; s, b) &= \frac{b}{s} \left( \left( \frac{y}{s} \right)^b - 1 \right) \\
\nabla_{b} (y; s, b) &= \left( 1 - \left( \frac{y}{s} \right)^b \right) \ln \left( \frac{y}{s} \right) + \frac{1}{b} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{s, s} (s, b) &= \frac{b^2}{s^2} \\
\mathcal{I}_{s, b} (s, b) &= - \frac{\psi_0(1) + 1}{s} \\
\mathcal{I}_{b, b} (s, b) &= \frac{\psi_0(1)^2 + 2 \psi_0(1) + \psi_1(1) + 1}{b^2} \\
\end{aligned}
$$

### Further Reading

- Tomanová, P. and Holý, V. (2021). Clustering of Arrivals in Queueing Systems: Autoregressive Conditional Duration Approach. *Central European Journal of Operations Research*, **29**(3), 859–874. doi: [10.1007/s10100-021-00744-7](https://doi.org/10.1007/s10100-021-00744-7).

# Real Data

## Asymmetric Laplace Distribution

### Mean-Scale Parametrization

#### Parameters

- Mean parameter $m \in \mathbb{R}$
- Scale parameter $s \in (0, \infty)$
- Asymmetry parameter $a \in (0, \infty)$

#### Density Function

$$
f(y | m, s, a) = \frac{1}{s \left( 1 / a + a\right)}  \exp \left\{- \frac{\lvert y - m \rvert}{s} a^{\mathrm{sign}(y - m)} \right\}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m + s (1 / a - a) \\
\mathrm{var}[Y] &= s^2 (1 / a^2 + a^2) \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s, a) &= \frac{\mathrm{sign}(y - m) a^{\mathrm{sign}(y - m)}}{s} \\
\nabla_{s} (y; m, s, a) &= \frac{\lvert y - m \rvert a^{\mathrm{sign}(y - m)}}{s^2} - \frac{1}{s} \\
\nabla_{a} (y; m, s, a) &= -\frac{(y - m) a^{\mathrm{sign}(y - m)}}{s} + \frac{1 - a^2}{a + a^3} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s, a) &= \frac{1}{s^2} \\
\mathcal{I}_{m, s} (m, s, a) &= 0 \\
\mathcal{I}_{m, a} (m, s, a) &= -\frac{2}{s (1 + a^2)} \\
\mathcal{I}_{s, s} (m, s, a) &= \frac{1}{s^2} \\
\mathcal{I}_{s, a} (m, s, a) &= -\frac{1}{s a} \frac{1 - a^2}{1 + a^2} \\
\mathcal{I}_{a, a} (m, s, a) &= \frac{1}{a^2} + \frac{4}{(1 + a^2)^2} \\
\end{aligned}
$$

## Generalized Error Distribution

### Mean-Scale Parametrization

#### Parameters

- Mean parameter $m \in \mathbb{R}$
- Scale parameter $s \in (0, \infty)$
- Shape parameter $b \in (0, \infty)$

#### Density Function

$$
f(y | m, s, b) = \frac{b}{2 s \Gamma \left( \frac{1}{b} \right)}  \exp \left(- \left(\frac{|y - m|}{s} \right)^b \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m \\
\mathrm{var}[Y] &= \frac{\Gamma \left( \frac{3}{b} \right)}{\Gamma \left( \frac{1}{b} \right)} s^2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s, b) &= \frac{b}{s^b}  |y - m|^{b - 1} \mathrm{sign}(y - m) \\
\nabla_{s} (y; m, s, b) &= \frac{b}{s^{b+1}} |y - m|^b - \frac{1}{s} \\
\nabla_{b} (y; m, s, b) &= \frac{1}{b} - \frac{1}{s^b} |y - m|^b \ln \left( \frac{|y - m|}{s} \right) + \frac{1}{b^2} \psi_0 \left( \frac{1}{b} \right) \\
\end{aligned}
$$

### Notes

- The Fisher information is approximated using a Riemann sum evaluated at 1 000 points.

## Laplace Distribution

### Mean-Scale Parametrization

#### Parameters

- Mean parameter $m \in \mathbb{R}$
- Scale parameter $s \in (0, \infty)$

#### Density Function

$$
f(y | m, s) = \frac{1}{2s} \exp \left\{- \frac{\lvert y - m \rvert}{s} \right\}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m \\
\mathrm{var}[Y] &= 2s^2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s) &= \frac{\mathrm{sign}(y - m)}{s} \\
\nabla_{s} (y; m, s) &= \frac{\lvert y - m \rvert}{s^2} - \frac{1}{s} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s) &= \frac{1}{s^2} \\
\mathcal{I}_{m, s} (m, s) &= 0 \\
\mathcal{I}_{s, s} (m, s) &= \frac{1}{s^2} \\
\end{aligned}
$$

## Logistic Distribution

### Mean-Scale Parametrization

#### Parameters

- Mean parameter $m \in \mathbb{R}$
- Scale parameter $s \in (0, \infty)$

#### Density Function

$$
f(y | m, s) = \frac{1}{s} \exp \left( - \frac{x - m}{s} \right) \left( 1 + \exp \left( - \frac{x - m}{s} \right) \right)^{-2}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m \\
\mathrm{var}[Y] &= \frac{\pi^2}{3} s^2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s) &= \frac{1}{s} \tanh \left( \frac{y - m}{2 s} \right) \\
\nabla_{s} (y; m, s) &= \frac{y - m}{s^2} \tanh \left( \frac{y - m}{2 s} \right) - \frac{1}{s} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s) &= \frac{1}{3 s^2} \\
\mathcal{I}_{m, s} (m, s) &= 0 \\
\mathcal{I}_{s, s} (m, s) &= \frac{1}{3 s^2} \left( \frac{\pi^2}{3} + 1 \right) \\
\end{aligned}
$$

## Normal Distribution

### Mean-Variance Parametrization

#### Parameters

- Mean parameter $m \in \mathbb{R}$
- Variance parameter $s \in (0, \infty)$

#### Density Function

$$
f(y | m, s) = \frac{1}{\sqrt{2 \pi s}} \exp \left( -\frac{(y - m)^2}{2 s} \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m \\
\mathrm{var}[Y] &= s \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s) &= \frac{y - m}{s} \\
\nabla_{s} (y; m, s) &= \frac{(y - m)^2 - s}{2 s^2} \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s) &= \frac{1}{s} \\
\mathcal{I}_{m, s} (m, s) &= 0 \\
\mathcal{I}_{s, s} (m, s) &= \frac{1}{2 s^2} \\
\end{aligned}
$$

## Student's t Distribution

### Mean-Variance Parametrization

#### Parameters

- Mean parameter $m \in \mathbb{R}$
- Variance parameter $s \in (0, \infty)$
- Degrees of freedom parameter $v \in (0, \infty)$

#### Density Function

$$
f(y | m, s, v) = \frac{\Gamma \left( \frac{v + 1}{2} \right)}{\Gamma \left( \frac{v}{2} \right) \sqrt{\pi s v}} \left( 1 + \frac{(y - m)^2}{s v} \right)^{-\frac{v + 1}{2}}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[Y] &= m, & \quad \text{for } v &> 1 \\
\mathrm{var}[Y] &= \frac{v}{v - 2} s, & \quad \text{for } v &> 2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{m} (y; m, s, v) &= \frac{(v + 1) (y - m) }{(y - m)^2 + s v} \\
\nabla_{s} (y; m, s, v) &= \frac{v}{2s} \frac{(y - m)^2 - s}{(y - m)^2 + s v} \\
\nabla_{v} (y; m, s, v) &= \frac{1}{2} \frac{(y - m)^2 - s}{(y - m)^2 + s v} - \frac{1}{2} \ln \left(1 + \frac{1}{v} \frac{(y - m)^2}{s} \right) - \frac{1}{2} \psi_0 \left( \frac{v}{2} \right) + \frac{1}{2} \psi_0 \left( \frac{v + 1}{2} \right) \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{m, m} (m, s, v) &= \frac{v + 1}{s (v + 3)} \\
\mathcal{I}_{m, s} (m, s, v) &= 0 \\
\mathcal{I}_{m, v} (m, s, v) &= 0 \\
\mathcal{I}_{s, s} (m, s, v) &= \frac{v}{2 s^2 (v + 3)} \\
\mathcal{I}_{s, v} (m, s, v) &= \frac{-1}{s (v + 1) (v + 3)} \\
\mathcal{I}_{v, v} (m, s, v) &= - \frac{1}{2} \frac{v + 5}{v (v + 1) (v + 3)} + \frac{1}{4} \psi_1 \left( \frac{v}{2} \right) - \frac{1}{4} \psi_1 \left( \frac{v + 1}{2} \right) \\
\end{aligned}
$$

### Further Reading

- Blazsek, S. and Villatoro, M. (2015). Is Beta-t-EGARCH(1,1) Superior to GARCH(1,1)? *Applied Economics*, **47**(17), 1764–1774. doi: [10.1080/00036846.2014.1000536](https://doi.org/10.1080/00036846.2014.1000536).

- Harvey, A. C. and Chakravarty, T. (2008). Beta-t-(E)GARCH. *Cambridge Working Papers in Economics*, CWPE 0840. doi: [10.17863/cam.5286](https://doi.org/10.17863/cam.5286).

- Harvey, A. C. and Lange, R. J. (2018). Modeling the Interactions Between Volatility and Returns using EGARCH-M. *Journal of Time Series Analysis*, **39**(6), 909–919. doi: [10.1111/jtsa.12419](https://doi.org/10.1111/jtsa.12419).

- Lange, K. L., Little, R. J. A., and Taylor, J. M. G. (1989). Robust Statistical Modeling Using the t Distribution. *Journal of the American Statistical Association*, **84**(408), 881–896. doi: [10.1080/01621459.1989.10478852](https://doi.org/10.1080/01621459.1989.10478852).

# Multivariate Real Data

## Multivariate Normal Distribution

### Mean-Variance Parametrization

#### Parameters

- Mean parameters $m_i \in \mathbb{R}, i = 1, \ldots, n$
- Variance parameters $s_i \in (0, \infty), i = 1, \ldots, n$
- Covariance parameters $c_{ij} \in \mathbb{R}, i = 2, \ldots, n, j = 1, \ldots, i$

#### Vector and Matrix Notation

- Mean vector $\boldsymbol{m}$ of length $n$
- Variance-covariance matrix $\boldsymbol{K}$ of size $n \times n$

#### Density Function

$$
f(\boldsymbol{y} | \boldsymbol{m}, \boldsymbol{K}) = \frac{1}{\sqrt{(2 \pi)^n | \boldsymbol{K}|}} \exp \left( - \frac{1}{2} (\boldsymbol{y} - \boldsymbol{m})^\intercal \boldsymbol{K}^{-1} (\boldsymbol{y} - \boldsymbol{m}) \right)
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[\boldsymbol{Y}] &= \boldsymbol{m} \\
\mathrm{var}[\boldsymbol{Y}] &= \boldsymbol{K} \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{\boldsymbol{m}} (\boldsymbol{y}; \boldsymbol{m}, \boldsymbol{K}) &= \boldsymbol{K}^{-1} \left(\boldsymbol{y} - \boldsymbol{m} \right) \\
\nabla_{\mathrm{vec}(\boldsymbol{K})} (\boldsymbol{y}; \boldsymbol{m}, \boldsymbol{K}) &= \mathrm{vec} \left( \frac{1}{2} \boldsymbol{K}^{-1} \left(\boldsymbol{y} - \boldsymbol{m} \right) \left(\boldsymbol{y} - \boldsymbol{m} \right)^\intercal \boldsymbol{K}^{-1} - \frac{1}{2} \boldsymbol{K}^{-1} \right) \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{\boldsymbol{m}, \boldsymbol{m}} (\boldsymbol{m}, \boldsymbol{K}) &= \boldsymbol{K}^{-1} \\
\mathcal{I}_{\boldsymbol{m}, \mathrm{vec}(\boldsymbol{K})} (\boldsymbol{m}, \boldsymbol{K}) &= \boldsymbol{0} \\
\mathcal{I}_{\mathrm{vec}(\boldsymbol{K}), \mathrm{vec}(\boldsymbol{K})} (\boldsymbol{m}, \boldsymbol{K}) &= \frac{1}{4} \boldsymbol{K}^{-1} \otimes \boldsymbol{K}^{-1} + \frac{1}{4} \mathrm{vec}\left(\boldsymbol{K}^{-1} \right) \mathrm{vec}\left(\boldsymbol{K}^{-1} \right)^\intercal \\
\end{aligned}
$$

## Multivariate Student's t Distribution

### Mean-Variance Parametrization

#### Parameters

- Mean parameters $m_i \in \mathbb{R}, i = 1, \ldots, n$
- Variance parameters $s_i \in (0, \infty), i = 1, \ldots, n$
- Covariance parameters $c_{ij} \in \mathbb{R}, i = 2, \ldots, n, j = 1, \ldots, i$
- Degrees of freedom parameter $v \in (0, \infty)$

#### Vector and Matrix Notation

- Mean vector $\boldsymbol{m}$ of length $n$
- Variance-covariance matrix $\boldsymbol{K}$ of size $n \times n$

#### Density Function

$$
f(\boldsymbol{y} | \boldsymbol{m}, \boldsymbol{K}, v) = \frac{\Gamma \left( \frac{v + n}{2} \right)}{\Gamma \left( \frac{v}{2} \right) \sqrt{(v \pi)^n | \boldsymbol{K}|}} \left( 1 + \frac{1}{v} (\boldsymbol{y} - \boldsymbol{m})^\intercal \boldsymbol{K}^{-1} (\boldsymbol{y} - \boldsymbol{m}) \right)^{-\frac{v + n}{2}}
$$

#### Moments

$$
\begin{aligned}
\mathrm{E}[\boldsymbol{Y}] &= \boldsymbol{m}, & \quad \text{for } v &> 1 \\
\mathrm{var}[\boldsymbol{Y}] &= \frac{v}{v - 2} \boldsymbol{K}, & \quad \text{for } v &> 2 \\
\end{aligned}
$$

#### Score

$$
\begin{aligned}
\nabla_{\boldsymbol{m}} (\boldsymbol{y}; \boldsymbol{m}, \boldsymbol{K}, v) &= \frac{v + n}{v + \left(\boldsymbol{y} - \boldsymbol{m} \right)^\intercal \boldsymbol{K}^{-1} \left(\boldsymbol{y} - \boldsymbol{m} \right)} \boldsymbol{K}^{-1} \left(\boldsymbol{y} - \boldsymbol{m} \right) \\
\nabla_{\mathrm{vec}(\boldsymbol{K})} (\boldsymbol{y}; \boldsymbol{m}, \boldsymbol{K}, v) &= 
\mathrm{vec} \left( \frac{1}{2} \frac{v + n}{v + \left(\boldsymbol{y} - \boldsymbol{m} \right)^\intercal \boldsymbol{K}^{-1} \left(\boldsymbol{y} - \boldsymbol{m} \right)} \boldsymbol{K}^{-1} \left(\boldsymbol{y} - \boldsymbol{m} \right) \left(\boldsymbol{y} - \boldsymbol{m} \right)^\intercal \boldsymbol{K}^{-1} - \frac{1}{2} \boldsymbol{K}^{-1} \right) \\
\nabla_{v} (\boldsymbol{y}; \boldsymbol{m}, \boldsymbol{K}, v) &= \frac{1}{2} \frac{ \left(\boldsymbol{y} - \boldsymbol{m} \right)^\intercal \boldsymbol{K}^{-1} \left(\boldsymbol{y} - \boldsymbol{m} \right) - n }{ \left(\boldsymbol{y} - \boldsymbol{m} \right)^\intercal \boldsymbol{K}^{-1} \left(\boldsymbol{y} - \boldsymbol{m} \right)) + v} - \frac{1}{2} \ln \left( 1 + \frac{1}{v} \left(\boldsymbol{y} - \boldsymbol{m} \right)^\intercal \boldsymbol{K}^{-1} \left(\boldsymbol{y} - \boldsymbol{m} \right) \right) \\
& \qquad - \frac{1}{2} \psi_0 \left( \frac{v}{2} \right) + \frac{1}{2} \psi_0 \left( \frac{v + n}{2} \right) \\
\end{aligned}
$$

#### Fisher Information

$$
\begin{aligned}
\mathcal{I}_{\boldsymbol{m}, \boldsymbol{m}} (\boldsymbol{m}, \boldsymbol{K}, v) &= \frac{v + n}{v + n + 2} \boldsymbol{K}^{-1} \\
\mathcal{I}_{\boldsymbol{m}, \mathrm{vec}(\boldsymbol{K})} (\boldsymbol{m}, \boldsymbol{K}, v) &= \boldsymbol{0} \\
\mathcal{I}_{\boldsymbol{m}, v} (\boldsymbol{m}, \boldsymbol{K}, v) &= \boldsymbol{0} \\
\mathcal{I}_{\mathrm{vec}(\boldsymbol{K}), \mathrm{vec}(\boldsymbol{K})} (\boldsymbol{m}, \boldsymbol{K}, v) &= 
\frac{1}{4} \frac{v + n}{v + n + 2} \boldsymbol{K}^{-1} \otimes \boldsymbol{K}^{-1} + \frac{1}{4} \frac{v + n - 2}{v + n + 2} \mathrm{vec}\left(\boldsymbol{K}^{-1} \right) \mathrm{vec}\left(\boldsymbol{K}^{-1} \right)^\intercal \\
\mathcal{I}_{\mathrm{vec}(\boldsymbol{K}), v} (\boldsymbol{m}, \boldsymbol{K}, v) &= - \frac{1}{(v + n +2)(v + n)} \mathrm{vec}\left(\boldsymbol{K}^{-1} \right) \\
\mathcal{I}_{v, v} (\boldsymbol{m}, \boldsymbol{K}, v) &= ) - \frac{1}{2} \frac{n (v + n + 4)}{v (v + n + 2)(v + n)} + \frac{1}{4} \psi_1 \left( \frac{v}{2} \right) - \frac{1}{4} \psi_1 \left( \frac{v + n}{2} \right) \\
\end{aligned}
$$

### Further Reading

- Blazsek, S. and Villatoro, M. (2015). Is Beta-t-EGARCH(1,1) Superior to GARCH(1,1)? *Applied Economics*, **47**(17), 1764–1774. doi: [10.1080/00036846.2014.1000536](https://doi.org/10.1080/00036846.2014.1000536).

- Harvey, A. C. and Chakravarty, T. (2008). Beta-t-(E)GARCH. *Cambridge Working Papers in Economics*, CWPE 0840. doi: [10.17863/cam.5286](https://doi.org/10.17863/cam.5286).

- Harvey, A. C. and Lange, R. J. (2018). Modeling the Interactions Between Volatility and Returns using EGARCH-M. *Journal of Time Series Analysis*, **39**(6), 909–919. doi: [10.1111/jtsa.12419](https://doi.org/10.1111/jtsa.12419).

- Lange, K. L., Little, R. J. A., and Taylor, J. M. G. (1989). Robust Statistical Modeling Using the t Distribution. *Journal of the American Statistical Association*, **84**(408), 881–896. doi: [10.1080/01621459.1989.10478852](https://doi.org/10.1080/01621459.1989.10478852).

