Calculus
Differentiation
Logarithmic Differentiation:
$$\dfrac{df(x)}{dx}=f(x)(\dfrac{d\ln f(x)}{dx})\text{, since }\dfrac{d\ln f(x)}{dx}=\dfrac{df(x)/dx}{f(x)}$$
Integration by Parts
Integration by Parts:
$$\int{udv}=uv-\int{vdu}$$
Tabular Method of Integration by Parts:
$$\begin{matrix} u & {} & dv \\ f & {} & g \\ {} & \overset{+}{\mathop{\searrow }}\, & {} \\ {{f}^{(1)}} & {} & {{g}^{(-1)}} \\ {} & \overset{-}{\mathop{\searrow }}\, & {} \\ {{f}^{(2)}} & {} & {{g}^{(-2)}} \\ {} & \overset{+}{\mathop{\searrow }}\, & {} \\ {{f}^{(3)}} & {} & {{g}^{(-3)}} \\ {} & \overset{-}{\mathop{\searrow }}\, & {} \\ {\vdots} & {} & {\vdots} \\ {f^{(n-1)}} & {} & {g^{-(n-1)}} \\ {} & {\overset{(-1)^{n-1}}{\mathop{\searrow }}} & {} \\ {f^{(n)}} & {} & {g^{(-n)}} \\ \end{matrix}$$
Mean
With PDF:
$$E[X]=\int_{-\infty}^{\infty}{xf(x)dx}$$
With Survival Function:
$$E[X]=\int_{0}^{\infty}{S(x)dx}=\int_{0}^{\infty}{(1-F(x))dx}$$
With a Function:
$$E[g(X)]=\int_{-\infty}^{\infty}{g(x)f_X(x)dx}$$ $$E[g(X)]=\int_{0}^{\infty}{g'(x)S_X(x)dx}\text{, for domain }x\ge 0$$ $$E[g(X)|j\le X\le k]=\dfrac{\int_{j}^{k}{g(x)f_X(x)dx}}{Pr(j\le X\le k)}$$
The Mean of Continuous R.V. and a Constant:
$$E[\min(X,k)]=\int_{-\infty}^{k}{x(x)dx}+k(1-F(k))$$
$$E[\min(X,k)]=\int_{-\infty}^{k}{x(x)dx}+k(1-F(k))$$
If \(f(x)=0\) for \(x<0\), then:
> \(E[X]=\int_{0}^{\infty}{(1-F(x))dx}\)
> \(E[\min(X,k)]=\int_{0}^{k}{(1-F(x))dx}\)
Variance and Other Moments
Variance:
$$Var(X)=E[X^{2}]-\mu^{2}$$
$$Var(aX+b)=a^2Var(X)$$
$$CV[X] = \dfrac{SD[X]}{E[X]}$$
Bernoulli Shortcut:
If \(Pr(X=a)=1-p\) and \(Pr(X=b)=p\), then \(Var(X)=(b-a)^{2}p(1-p)\)
Covariance
$$Cov(𝑋,Y)=𝐸[(X-\mu_X)(Y-\mu_Y)]$$
$$Var(X+Y)=Var(X)+Var(Y)+2Cov(X,Y)$$
$$Var(aX+bY)=a^2Var(X)+b^2Var(Y)+2abCov(X,Y)$$
If two variables are independent, then the expected value of their product equals the product of their expectations.
$$E[XY]=E[X]E[Y]$$
Note: Independence implies covariance of 0, but not conversely.
Covariance Rules:
$$Cov(X,X)=Var(X)$$
$$Cov(X,Y)=Cov(Y,X)$$
$$Cov(aX,bY)=abCov(X,Y)$$
$$Cov(X,aY+bZ)=aCov(X,Y)+bCov(X,Z)$$
Covariance Matrix:
$$\sum =(\begin{matrix}
\sigma _{X}^{2} & {{\sigma }_{XY}} \\
{{\sigma }_{XY}} & \sigma _{Y}^{2} \\
\end{matrix})$$
Correlation Coefficient:
$$\rho_{X,Y}=Corr(X,Y)=\dfrac{Cov(X,Y)}{\sqrt{Var(X)}\sqrt{Var(Y)}}$$
Joint and Marginal Distributions
Joint Density Function:
$$Pr(a < X {\leq } b,c < Y {\leq } d)=\int_{a}^{b}{\int_{c}^{d}{f(x,y)dydx}}$$
Marginal Distribution:
$$f_X(x)=\int_{-\infty}^{\infty}{f(x,y)dy}$$ $$f_Y(y)=\int_{-\infty}^{\infty}{f(x,y)dx}$$
If the variables are independent, then:
$$f(x,y)=g(x)h(y)$$
Joint Moments:
$$E[g(x,y)]=\int{\int{g(x,y)f(x,y)dydx}}$$
If the variables are independent, then:
$$E[g(X)h(Y)] = E[g(X)]E[h(Y)]$$
Conditional Distributions
Conditional Density:
$$f_{X|Y}(x|y)=\dfrac{f(x,y)}{f_Y(y)}$$
Law of Total Probabilities:
$$f_X(x)=\int_{-\infty}^{\infty}{f_{X|Y}(x|y)f_Y(y)dy}$$
Bayes' Theorem:
$$f_{X|Y}(x|y)=\dfrac{f_{X|Y}(x|y)f_Y(y)dy}{f_X(x)}=\dfrac{f_{X|Y}(x|y)f_Y(y)dy}{\int_{-\infty}^{\infty}{f_{Y|X}(y|w)f_X(w)dw}}$$
Conditional Moments:
$$E[g(X)|Y=y]=\int_{-\infty}^{\infty}{\dfrac{g(x)f(x,y)dx}{f_Y(y)}}$$
Double Expectation:
$$E[g(X)]=E_Y[E_X[g(X)|Y]]$$
Specifically,
> \(E[g(X)]=E_Y[E_X[X|Y]]\)
> \(E[g(X^2)]=E_Y[E_X[X^2|Y]]\)
Conditional Variance:
$$Var(X)=E[Var(X|Y)]+Var(E[X|Y])$$
Order Statistics
Maximum & Minimum:
$$X_{(1)}=min(X_1,X_2,...,X_n)$$
$$X_{(n)}=max(X_1,X_2,...,X_n)$$
For IID r.v's:
For minimum,
$$S_{X_{(1)}}(x)=[S_X(x)]^n\text{, and the pdf is }f_{X_{(1)}}(x)=f_Y(x)=n(1-F_X(x))^{n-1}f_X(x)$$
For Maximum,
$$F_{X_{(n)}}(x)=[F_X(x)]^n\text{, and the pdf is }f_{X_{(n)}}(x)=f_Y(x)=n(F_X(x))^{n-1}f_X(x)$$
\(k^{th}\) Order Statistic:
$$f_{Y_{(k)}}(x)=\dfrac{n!}{(k-1)!1!(n-k)!}F_X(x)^{k-1}f_X(x)(1-F_X(x))^{n-k}$$
For Uniform r.v. on \([0,\theta]\),
$$E_{Y_{(k)}}(x)=\dfrac{k\theta}{n+1}$$