org.hipparchus.stat.regression.AbstractMultipleLinearRegression

org.hipparchus.stat.regression.OLSMultipleLinearRegression

All Implemented Interfaces:: MultipleLinearRegression

public class OLSMultipleLinearRegression extends AbstractMultipleLinearRegression

Implements ordinary least squares (OLS) to estimate the parameters of a multiple linear regression model.

The regression coefficients, b, satisfy the normal equations:

 X^T X b = X^T y

To solve the normal equations, this implementation uses QR decomposition of the X matrix. (See QRDecomposition for details on the decomposition algorithm.) The X matrix, also known as the design matrix, has rows corresponding to sample observations and columns corresponding to independent variables. When the model is estimated using an intercept term (i.e. when isNoIntercept is false as it is by default), the X matrix includes an initial column identically equal to 1. We solve the normal equations as follows:

 X^TX b = X^T y
 (QR)^T (QR) b = (QR)^Ty
 R^T (Q^TQ) R b = R^T Q^T y
 R^T R b = R^T Q^T y
 (R^T)^-1 R^T R b = (R^T)^-1 R^T Q^T y
 R b = Q^T y

Given Q and R, the last equation is solved by back-substitution.

Constructor Summary

Constructors

Constructor

Description

OLSMultipleLinearRegression()

Create an empty OLSMultipleLinearRegression instance.

OLSMultipleLinearRegression(double threshold)

Create an empty OLSMultipleLinearRegression instance, using the given singularity threshold for the QR decomposition.
Method Summary

Modifier and Type

Method

Description

double

calculateAdjustedRSquared()

Returns the adjusted R-squared statistic, defined by the formula \(R_\mathrm{adj}^2 = 1 - \frac{\mathrm{SSR} (n - 1)}{\mathrm{SSTO} (n - p)}\) where SSR is the sum of squared residuals, SSTO is the total sum of squares, n is the number of observations and p is the number of parameters estimated (including the intercept).

protected RealVector

calculateBeta()

Calculates the regression coefficients using OLS.

protected RealMatrix

calculateBetaVariance()

Calculates the variance-covariance matrix of the regression parameters.

RealMatrix

calculateHat()

Compute the "hat" matrix.

double

calculateResidualSumOfSquares()

Returns the sum of squared residuals.

double

calculateRSquared()

Returns the R-Squared statistic, defined by the formula \(R^2 = 1 - \frac{\mathrm{SSR}}{\mathrm{SSTO}}\) where SSR is the sum of squared residuals and SSTO is the total sum of squares

double

calculateTotalSumOfSquares()

Returns the sum of squared deviations of Y from its mean.

void

newSampleData(double[] y, double[][] x)

Loads model x and y sample data, overriding any previous sample.

void

newSampleData(double[] data, int nobs, int nvars)

Loads model x and y sample data from a flat input array, overriding any previous sample.

protected void

newXSampleData(double[][] x)

Loads new x sample data, overriding any previous data.

Methods inherited from class org.hipparchus.stat.regression.AbstractMultipleLinearRegression
calculateErrorVariance, calculateResiduals, calculateYVariance, estimateErrorVariance, estimateRegressandVariance, estimateRegressionParameters, estimateRegressionParametersStandardErrors, estimateRegressionParametersVariance, estimateRegressionStandardError, estimateResiduals, getX, getY, isNoIntercept, newYSampleData, setNoIntercept, validateCovarianceData, validateSampleData

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details
- OLSMultipleLinearRegression
  
  public OLSMultipleLinearRegression()
  
  Create an empty OLSMultipleLinearRegression instance.
- OLSMultipleLinearRegression
  
  public OLSMultipleLinearRegression(double threshold)
  
  Create an empty OLSMultipleLinearRegression instance, using the given singularity threshold for the QR decomposition.
  
  Parameters:
  
  threshold - the singularity threshold
Method Details
- newSampleData
  
  public void newSampleData(double[] y, double[][] x) throws MathIllegalArgumentException
  
  Loads model x and y sample data, overriding any previous sample. Computes and caches QR decomposition of the X matrix.
  
  Parameters:
  
  y - the [n,1] array representing the y sample
  
  x - the [n,k] array representing the x sample
  
  Throws:
  
  MathIllegalArgumentException - if the x and y array data are not compatible for the regression
- newSampleData
  
  public void newSampleData(double[] data, int nobs, int nvars)
  Loads model x and y sample data from a flat input array, overriding any previous sample.
  
  Assumes that rows are concatenated with y values first in each row. For example, an input data array containing the sequence of values (1, 2, 3, 4, 5, 6, 7, 8, 9) with nobs = 3 and nvars = 2 creates a regression dataset with two independent variables, as below:
  
  y x[0] x[1] -------------- 1 2 3 4 5 6 7 8 9
  
  Note that there is no need to add an initial unitary column (column of 1's) when specifying a model including an intercept term. If AbstractMultipleLinearRegression.isNoIntercept() is true, the X matrix will be created without an initial column of "1"s; otherwise this column will be added.
  
  Throws IllegalArgumentException if any of the following preconditions fail:
  
  data cannot be null
  
  data.length = nobs * (nvars + 1)
  
  nobs > nvars
  
  This implementation computes and caches the QR decomposition of the X matrix.
  Overrides:
  
  newSampleData in class AbstractMultipleLinearRegression
  
  Parameters:
  
  data - input data array
  
  nobs - number of observations (rows)
  
  nvars - number of independent variables (columns, not counting y)
- calculateHat
  
  public RealMatrix calculateHat()
  
  Compute the "hat" matrix.
  
  The hat matrix is defined in terms of the design matrix X by X(X^TX)^-1X^T
  
  The implementation here uses the QR decomposition to compute the hat matrix as Q I_pQ^T where I_p is the p-dimensional identity matrix augmented by 0's. This computational formula is from "The Hat Matrix in Regression and ANOVA", David C. Hoaglin and Roy E. Welsch, The American Statistician, Vol. 32, No. 1 (Feb., 1978), pp. 17-22.
  
  Data for the model must have been successfully loaded using one of the newSampleData methods before invoking this method; otherwise a NullPointerException will be thrown.
  
  Returns:
  
  the hat matrix
  
  Throws:
  
  NullPointerException - unless method newSampleData has been called beforehand.
- calculateTotalSumOfSquares
  
  public double calculateTotalSumOfSquares()
  
  Returns the sum of squared deviations of Y from its mean.
  
  If the model has no intercept term, 0 is used for the mean of Y - i.e., what is returned is the sum of the squared Y values.
  
  The value returned by this method is the SSTO value used in the R-squared computation.
  Returns:
  
  SSTO - the total sum of squares
  
  Throws:
  
  NullPointerException - if the sample has not been set
  
  See Also:
  
  AbstractMultipleLinearRegression.isNoIntercept()
- calculateResidualSumOfSquares
  
  public double calculateResidualSumOfSquares()
  
  Returns the sum of squared residuals.
  
  Returns:
  
  residual sum of squares
  
  Throws:
  
  MathIllegalArgumentException - if the design matrix is singular
  
  NullPointerException - if the data for the model have not been loaded
- calculateRSquared
  
  public double calculateRSquared()
  
  Returns the R-Squared statistic, defined by the formula \(R^2 = 1 - \frac{\mathrm{SSR}}{\mathrm{SSTO}}\) where SSR is the sum of squared residuals and SSTO is the total sum of squares
  If there is no variance in y, i.e., SSTO = 0, NaN is returned.
  
  Returns:
  
  R-square statistic
  
  Throws:
  
  NullPointerException - if the sample has not been set
  
  MathIllegalArgumentException - if the design matrix is singular
- calculateAdjustedRSquared
  
  public double calculateAdjustedRSquared()
  Returns the adjusted R-squared statistic, defined by the formula \(R_\mathrm{adj}^2 = 1 - \frac{\mathrm{SSR} (n - 1)}{\mathrm{SSTO} (n - p)}\) where SSR is the sum of squared residuals, SSTO is the total sum of squares, n is the number of observations and p is the number of parameters estimated (including the intercept).
  
  If the regression is estimated without an intercept term, what is returned is
  1 - (1 - calculateRSquared()) * (n / (n - p))
  
  If there is no variance in y, i.e., SSTO = 0, NaN is returned.
  Returns:
  
  adjusted R-Squared statistic
  
  Throws:
  
  NullPointerException - if the sample has not been set
  
  MathIllegalArgumentException - if the design matrix is singular
  
  See Also:
  
  AbstractMultipleLinearRegression.isNoIntercept()
- newXSampleData
  
  protected void newXSampleData(double[][] x)
  Loads new x sample data, overriding any previous data.
  
  The input x array should have one row for each sample observation, with columns corresponding to independent variables. For example, if
  
  x = new double[][] {{1, 2}, {3, 4}, {5, 6}}
  
  then setXSampleData(x) results in a model with two independent variables and 3 observations:
  
  x[0] x[1] ---------- 1 2 3 4 5 6
  
  Note that there is no need to add an initial unitary column (column of 1's) when specifying a model including an intercept term.
  
  This implementation computes and caches the QR decomposition of the X matrix once it is successfully loaded.
  Overrides:
  
  newXSampleData in class AbstractMultipleLinearRegression
  
  Parameters:
  
  x - the rectangular array representing the x sample
- calculateBeta
  
  protected RealVector calculateBeta()
  
  Calculates the regression coefficients using OLS.
  Data for the model must have been successfully loaded using one of the newSampleData methods before invoking this method; otherwise a NullPointerException will be thrown.
  
  Specified by:
  
  calculateBeta in class AbstractMultipleLinearRegression
  
  Returns:
  
  beta
  
  Throws:
  
  MathIllegalArgumentException - if the design matrix is singular
  
  NullPointerException - if the data for the model have not been loaded
- calculateBetaVariance
  
  protected RealMatrix calculateBetaVariance()
  
  Calculates the variance-covariance matrix of the regression parameters.
  
  Var(b) = (X^TX)^-1
  
  Uses QR decomposition to reduce (X^TX)^-1 to (R^TR)^-1, with only the top p rows of R included, where p = the length of the beta vector.
  
  Data for the model must have been successfully loaded using one of the newSampleData methods before invoking this method; otherwise a NullPointerException will be thrown.
  
  Specified by:
  
  calculateBetaVariance in class AbstractMultipleLinearRegression
  
  Returns:
  
  The beta variance-covariance matrix
  
  Throws:
  
  MathIllegalArgumentException - if the design matrix is singular
  
  NullPointerException - if the data for the model have not been loaded

Class OLSMultipleLinearRegression

Constructor Summary

Method Summary

Methods inherited from class org.hipparchus.stat.regression.AbstractMultipleLinearRegression

Methods inherited from class java.lang.Object

Constructor Details

OLSMultipleLinearRegression

OLSMultipleLinearRegression

Method Details

newSampleData

newSampleData

calculateHat

calculateTotalSumOfSquares

calculateResidualSumOfSquares

calculateRSquared

calculateAdjustedRSquared

newXSampleData

calculateBeta

calculateBetaVariance