Logarithmic Differentiation:

$$\dfrac{df(x)}{dx}=f(x)(\dfrac{d\ln f(x)}{dx})\text{, since }\dfrac{d\ln f(x)}{dx}=\dfrac{df(x)/dx}{f(x)}$$

Integration by Parts:

$$\int{udv}=uv-\int{vdu}$$

Tabular Method of Integration by Parts:

$$\begin{matrix} u & {} & dv \\ f & {} & g \\ {} & \overset{+}{\mathop{\searrow }}\, & {} \\ {{f}^{(1)}} & {} & {{g}^{(-1)}} \\ {} & \overset{-}{\mathop{\searrow }}\, & {} \\ {{f}^{(2)}} & {} & {{g}^{(-2)}} \\ {} & \overset{+}{\mathop{\searrow }}\, & {} \\ {{f}^{(3)}} & {} & {{g}^{(-3)}} \\ {} & \overset{-}{\mathop{\searrow }}\, & {} \\ {\vdots} & {} & {\vdots} \\ {f^{(n-1)}} & {} & {g^{-(n-1)}} \\ {} & {\overset{(-1)^{n-1}}{\mathop{\searrow }}} & {} \\ {f^{(n)}} & {} & {g^{(-n)}} \\ \end{matrix}$$

With PDF:

$$E[X]=\int_{-\infty}^{\infty}{xf(x)dx}$$

With Survival Function:

$$E[X]=\int_{0}^{\infty}{S(x)dx}=\int_{0}^{\infty}{(1-F(x))dx}$$

With a Function:

$$E[g(X)]=\int_{-\infty}^{\infty}{g(x)f_X(x)dx}$$ $$E[g(X)]=\int_{0}^{\infty}{g'(x)S_X(x)dx}\text{, for domain }x\ge 0$$ $$E[g(X)|j\le X\le k]=\dfrac{\int_{j}^{k}{g(x)f_X(x)dx}}{Pr(j\le X\le k)}$$

The Mean of Continuous R.V. and a Constant:

$$E[\min(X,k)]=\int_{-\infty}^{k}{x(x)dx}+k(1-F(k))$$ $$E[\min(X,k)]=\int_{-\infty}^{k}{x(x)dx}+k(1-F(k))$$ If \(f(x)=0\) for \(x<0\), then:

    > \(E[X]=\int_{0}^{\infty}{(1-F(x))dx}\)

    > \(E[\min(X,k)]=\int_{0}^{k}{(1-F(x))dx}\)

Variance:

$$Var(X)=E[X^{2}]-\mu^{2}$$ $$Var(aX+b)=a^2Var(X)$$ $$CV[X] = \dfrac{SD[X]}{E[X]}$$

Bernoulli Shortcut:

If \(Pr(X=a)=1-p\) and \(Pr(X=b)=p\), then \(Var(X)=(b-a)^{2}p(1-p)\)

$$Cov(𝑋,Y)=𝐸[(X-\mu_X)(Y-\mu_Y)]$$ $$Var(X+Y)=Var(X)+Var(Y)+2Cov(X,Y)$$ $$Var(aX+bY)=a^2Var(X)+b^2Var(Y)+2abCov(X,Y)$$

If two variables are independent, then the expected value of their product equals the product of their expectations.

$$E[XY]=E[X]E[Y]$$

Note: Independence implies covariance of 0, but not conversely.

Covariance Rules:

$$Cov(X,X)=Var(X)$$ $$Cov(X,Y)=Cov(Y,X)$$ $$Cov(aX,bY)=abCov(X,Y)$$ $$Cov(X,aY+bZ)=aCov(X,Y)+bCov(X,Z)$$

Covariance Matrix:

$$\sum =(\begin{matrix} \sigma _{X}^{2} & {{\sigma }_{XY}} \\ {{\sigma }_{XY}} & \sigma _{Y}^{2} \\ \end{matrix})$$

Correlation Coefficient:

$$\rho_{X,Y}=Corr(X,Y)=\dfrac{Cov(X,Y)}{\sqrt{Var(X)}\sqrt{Var(Y)}}$$

Joint Density Function:

$$Pr(a < X {\leq } b,c < Y {\leq } d)=\int_{a}^{b}{\int_{c}^{d}{f(x,y)dydx}}$$

Marginal Distribution:

$$f_X(x)=\int_{-\infty}^{\infty}{f(x,y)dy}$$ $$f_Y(y)=\int_{-\infty}^{\infty}{f(x,y)dx}$$

If the variables are independent, then:

$$f(x,y)=g(x)h(y)$$

Joint Moments:

$$E[g(x,y)]=\int{\int{g(x,y)f(x,y)dydx}}$$

If the variables are independent, then:

$$E[g(X)h(Y)] = E[g(X)]E[h(Y)]$$

Conditional Density:

$$f_{X|Y}(x|y)=\dfrac{f(x,y)}{f_Y(y)}$$

Law of Total Probabilities:

$$f_X(x)=\int_{-\infty}^{\infty}{f_{X|Y}(x|y)f_Y(y)dy}$$

Bayes' Theorem:

$$f_{X|Y}(x|y)=\dfrac{f_{X|Y}(x|y)f_Y(y)dy}{f_X(x)}=\dfrac{f_{X|Y}(x|y)f_Y(y)dy}{\int_{-\infty}^{\infty}{f_{Y|X}(y|w)f_X(w)dw}}$$

Conditional Moments:

$$E[g(X)|Y=y]=\int_{-\infty}^{\infty}{\dfrac{g(x)f(x,y)dx}{f_Y(y)}}$$

Double Expectation:

$$E[g(X)]=E_Y[E_X[g(X)|Y]]$$

Specifically,

    > \(E[g(X)]=E_Y[E_X[X|Y]]\)

    > \(E[g(X^2)]=E_Y[E_X[X^2|Y]]\)

Conditional Variance:

$$Var(X)=E[Var(X|Y)]+Var(E[X|Y])$$

Maximum & Minimum:

$$X_{(1)}=min(X_1,X_2,...,X_n)$$ $$X_{(n)}=max(X_1,X_2,...,X_n)$$

For IID r.v's:

For minimum,

$$S_{X_{(1)}}(x)=[S_X(x)]^n\text{, and the pdf is }f_{X_{(1)}}(x)=f_Y(x)=n(1-F_X(x))^{n-1}f_X(x)$$

For Maximum,

$$F_{X_{(n)}}(x)=[F_X(x)]^n\text{, and the pdf is }f_{X_{(n)}}(x)=f_Y(x)=n(F_X(x))^{n-1}f_X(x)$$

\(k^{th}\) Order Statistic:

$$f_{Y_{(k)}}(x)=\dfrac{n!}{(k-1)!1!(n-k)!}F_X(x)^{k-1}f_X(x)(1-F_X(x))^{n-k}$$

For Uniform r.v. on \([0,\theta]\),

$$E_{Y_{(k)}}(x)=\dfrac{k\theta}{n+1}$$