Skip to contents

# Collection of Machine Learning Model Metrics for Easy Reference

Source: `R/MLMetrics.R`
`MLMetrics.Rd`

This function estimates over 40 Metrics for assessing the quality of Machine Learning Models. The purpose is to provide a wrapper which brings all the metrics on the table and makes it easier to use them to select a model.

## Usage

    MLMetrics(Observed, yvalue, Model, K, Name, Form, kutuf, TTy)

## Arguments

Observed

  The Observed data in a data frame format

yvalue

  The Response variable of the estimated Model

Model

  The Estimated Model (Model = a + bx)

K

  The number of variables in the estimated Model to consider

Name

  The Name of the Models that need to be specified. They are ARIMA, Values if the model computes the fitted value without estimation like Essembles, SMOOTH (smooth.spline), Logit, Ensembles based on weight - EssemWet, QUADRATIC polynomial, SPLINE polynomial.

Form

  Form of the Model Estimated (LM, ALM, GLM, N-LM, ARDL)

kutuf

  Cutoff for the Estimated values (defaults to 0.5 if not specified)

TTy

  Type of response variable (Numeric or Response - like binary)

## Value

A list with the following components:

`Absolute Error`

  of the Model.

`Absolute Percent Error`

  of the Model.

`Accuracy`

  of the Model.

`Adjusted R Square`

  of the Model.

``Akaike's` Information Criterion AIC`

  of the Model.

`Area under the ROC curve (AUC)`

  of the Model.

`Average Precision at k`

  of the Model.

`Bias`

  of the Model.

`Brier score`

  of the Model.

`Classification Error`

  of the Model.

`F1 Score`

  of the Model.

`fScore`

  of the Model.

`GINI Coefficient`

  of the Model.

`kappa statistic`

  of the Model.

`Log Loss`

  of the Model.

``Mallow's` cp`

  of the Model.

`Matthews Correlation Coefficient`

  of the Model.

`Mean Log Loss`

  of the Model.

`Mean Absolute Error`

  of the Model.

`Mean Absolute Percent Error`

  of the Model.

`Mean Average Precision at k`

  of the Model.

`Mean Absolute Scaled Error`

  of the Model.

`Median Absolute Error`

  of the Model.

`Mean Squared Error`

  of the Model.

`Mean Squared Log Error`

  of the Model.

`Model turning point error`

  of the Model.

`Negative Predictive Value`

  of the Model.

`Percent Bias`

  of the Model.

`Positive Predictive Value`

  of the Model.

`Precision`

  of the Model.

`R Square`

  of the Model.

`Relative Absolute Error`

  of the Model.

`Recall`

  of the Model.

`Root Mean Squared Error`

  of the Model.

`Root Mean Squared Log Error`

  of the Model.

`Root Relative Squared Error`

  of the Model.

`Relative Squared Error`

  of the Model.

``Schwarz's` Bayesian criterion BIC`

  of the Model.

`Sensitivity`

  of the Model.

`specificity`

  of the Model.

`Squared Error`

  of the Model.

`Squared Log Error`

  of the Model.

`Symmetric Mean Absolute Percentage Error`

  of the Model.

`Sum of Squared Errors`

  of the Model.

`True negative rate`

  of the Model.

`True positive rate`

  of the Model.

## Examples

    library(splines)
    Model   <- lm(states ~ bs(sequence, knots = c(30, 115)), data = Data)
    MLMetrics(Observed = Data, yvalue = Data$states, Model = Model, K = 2,
     Name = "Linear", Form = "LM", kutuf = 0, TTy = "Number")
    #> Warning: NaNs produced
    #> Warning: actual should be a list of vectors. Converting to a list.
    #> Warning: predicted should be a list of vectors. Converting to a list.
    #> $`Absolute Error`
    #> [1] 460
    #>
    #> $`Absolute Percent Error`
    #> [1] 51
    #>
    #> $Accuracy
    #> [1] 0
    #>
    #> $`Adjusted R Square`
    #> [1] 0.77
    #>
    #> $`Akaike's Information Criterion AIC`
    #> [1] 1000
    #>
    #> $`Area under the ROC curve (AUC)`
    #> [1] 0
    #>
    #> $`Average Precision at k`
    #> [1] 0
    #>
    #> $Bias
    #> [1] 0.00000000000000017
    #>
    #> $`Brier score`
    #> [1] 8
    #>
    #> $`Classification Error`
    #> [1] 1
    #>
    #> $`F1 Score`
    #> [1] 0
    #>
    #> $fScore
    #> [1] 0
    #>
    #> $`GINI Coefficient`
    #> [1] 0.8
    #>
    #> $`kappa statistic`
    #> [1] 0
    #>
    #> $`Log Loss`
    #> [1] Inf
    #>
    #> $`Mallow's cp`
    #> [1] 3
    #>
    #> $`Matthews Correlation Coefficient`
    #> [1] 0
    #>
    #> $`Mean Log Loss`
    #> [1] -480
    #>
    #> $`Mean Absolute Error`
    #> [1] 2.3
    #>
    #> $`Mean Absolute Percent Error`
    #> [1] 0.25
    #>
    #> $`Mean Average Precision at k`
    #> [1] 0
    #>
    #> $`Mean Absolute Scaled Error`
    #> [1] 0.74
    #>
    #> $`Median Absolute Error`
    #> [1] 1.9
    #>
    #> $`Mean Squared Error`
    #> [1] 8.4
    #>
    #> $`Mean Squared Log Error`
    #> [1] 0.072
    #>
    #> $`Model turning point error`
    #> [1] 110
    #>
    #> $`Negative Predictive Value`
    #> [1] 0
    #>
    #> $`Percent Bias`
    #> [1] -0.1
    #>
    #> $`Positive Predictive Value`
    #> [1] 0
    #>
    #> $Precision
    #> [1] 1
    #>
    #> $`R Square`
    #> [1] 0.78
    #>
    #> $`Relative Absolute Error`
    #> [1] 0.47
    #>
    #> $Recall
    #> [1] 1
    #>
    #> $`Root Mean Squared Error`
    #> [1] 2.9
    #>
    #> $`Root Mean Squared Log Error`
    #> [1] 0.27
    #>
    #> $`Root Relative Squared Error`
    #> [1] 0.47
    #>
    #> $`Relative Squared Error`
    #> [1] 0.22
    #>
    #> $`Schwarz's Bayesian criterion BIC`
    #> [1] 1000
    #>
    #> $Sensitivity
    #> [1] 0
    #>
    #> $specificity
    #> [1] 0
    #>
    #> $`Squared Error`
    #> [1] 1700
    #>
    #> $`Squared Log Error`
    #> [1] 14
    #>
    #> $`Symmetric Mean Absolute Percentage Error`
    #> [1] 0.21
    #>
    #> $`Sum of Squared Errors`
    #> [1] 1700
    #>
    #> $`True negative rate`
    #> [1] 0
    #>
    #> $`True positive rate`
    #> [1] 0
    #>