Type: | Package |
Title: | Multivariate Version of the Diebold-Mariano Test |
Version: | 1.1.5 |
Imports: | MTS |
Date: | 2025-03-08 |
Author: | Krzysztof Drachal [aut, cre] (Faculty of Economic Sciences, University of Warsaw, Poland) |
Maintainer: | Krzysztof Drachal <kdrachal@wne.uw.edu.pl> |
Description: | Allows to perform the multivariate version of the Diebold-Mariano test for equal predictive ability of multiple forecast comparison. Main reference: Mariano, R.S., Preve, D. (2012) <doi:10.1016/j.jeconom.2012.01.014>. |
License: | GPL-3 |
LazyData: | TRUE |
URL: | https://CRAN.R-project.org/package=multDM |
Note: | Research funded by the Polish National Science Centre grant under the contract number DEC-2015/19/N/HS4/00205. |
NeedsCompilation: | no |
Packaged: | 2025-03-08 21:16:46 UTC; Krzysiek |
Repository: | CRAN |
Date/Publication: | 2025-03-08 21:30:02 UTC |
Computes Diebold-Mariano Test for the Equal Predictive Accuracy.
Description
This function computes Diebold-Mariano test for the equal predictive accuracy. The null hypothesis of this test is that two forecasts have the same accuracy. The alternative hypothesis can be specified as ”Both forecasts have different accuracy”, ”The first forecast is less accurate than the second forecast”, or ”The first forecast is more accurate than the second forecast”.
Usage
DM.test(f1,f2,y,loss.type="SE",h,c=FALSE,H1="same")
Arguments
f1 |
|
f2 |
|
y |
|
loss.type |
method to compute the loss function, |
h |
|
c |
|
H1 |
alternative hypothesis, |
Value
class htest
object, list
of
statistic |
test statistic |
parameter |
|
alternative |
alternative hypothesis of the test |
p.value |
p-value |
method |
name of the test |
data.name |
names of the tested time-series |
References
Diebold, F.X., Mariano, R. 1995. Comparing predictive accuracy. Journal of Business and Economic Statistics 13, 253–265.
Harvey, D., Leybourne, S., Newbold, P., 1997. Testing the equality of prediction mean squared errors. International Journal of Forecasting 13, 281–291.
Hyndman, R.J., Koehler, A.B. 2006. Another look at measures of forecast accuracy. International Journal of Forecasting 22, 679–688.
Taylor, S. J., 2005. Asset Price Dynamics, Volatility, and Prediction, Princeton University Press.
Triacca, U., 2018. Comparing Predictive Accuracy of Two Forecasts, https://www.lem.sssup.it/phd/documents/Lesson19.pdf.
Examples
data(MDMforecasts)
ts <- MDMforecasts$ts
forecasts <- MDMforecasts$forecasts
DM.test(f1=forecasts[,1],f2=forecasts[,2],y=ts,loss="SE",h=1,c=FALSE,H1="same")
Selects Models with Outstanding Predictive Ability basing on Multivariate Diebold-Mariano Test.
Description
This function selects models with outstanding predictive ability basing on multivariate Diebold-Mariano test MDM.test
.
Usage
MDM.selection(realized,evaluated,q,alpha,statistic="Sc",loss.type="SE")
Arguments
realized |
|
evaluated |
|
q |
|
alpha |
|
statistic |
|
loss.type |
method to compute the loss function, |
Value
class MDM
object, list
of
outcomes |
|
p.value |
|
alpha |
|
eliminated |
|
References
Mariano R.S., Preve, D., 2012. Statistical tests for multiple forecast comparison. Journal of Econometrics 169, 123–130.
Examples
data(MDMforecasts)
ts <- MDMforecasts$ts
forecasts <- MDMforecasts$forecasts
MDM.selection(realized=ts,evaluated=forecasts,q=10,alpha=0.1,statistic="S",loss.type="AE")
Computes Multivariate Diebold-Mariano Test for the Equal Predictive Accuracy of Two or More Non-nested Forecasting Models.
Description
This function computes multivariate Diebold-Mariano test for the equal predictive accuracy of two or more non-nested forecasting models. The null hypothesis of this test is that the evaluated forecasts have the same accuracy. The alternative hypothesis is that Equal predictive accuracy (EPA) does not hold.
Usage
MDM.test(realized,evaluated,q,statistic="Sc",loss.type="SE")
Arguments
realized |
|
evaluated |
|
q |
|
statistic |
|
loss.type |
method to compute the loss function, |
Value
class htest
object, list
of
statistic |
test statistic |
parameter |
|
alternative |
alternative hypothesis of the test |
p.value |
p-value |
method |
name of the test |
data.name |
names of the tested objects |
References
Mariano R.S., Preve, D., 2012. Statistical tests for multiple forecast comparison. Journal of Econometrics 169, 123–130.
Examples
data(MDMforecasts)
ts <- MDMforecasts$ts
forecasts <- MDMforecasts$forecasts
MDM.test(realized=ts,evaluated=forecasts,q=10,statistic="S",loss.type="AE")
Sample Data.
Description
Sample artificial data.
Usage
data(MDMforecasts)
Format
MDMforecasts
is list
object such that
-
MDMforecasts$ts
isvector
of time-series which is of interest to model -
MDMforecasts$forecasts
ismatrix
of 20 different forecasts ofts
from 20 different forecasting models, each row represents different forecast and time is indexed by columns
Examples
data(MDMforecasts)
ts <- MDMforecasts$ts
forecasts <- MDMforecasts$forecasts
MDM.test(realized=ts,evaluated=forecasts,q=10,statistic="S",loss.type="AE")
Computes Tiao-Box Test for Autocorrelation.
Description
This function computes Tiao-Box test for autocorrelation, i.e, coefficient of p
-th lag in VAR(p) model. Its null hypothesis is that p
-th lag is not essential. The alternative hypothesis is that it is essential.
Usage
TB_AR_test(d,p)
Arguments
d |
|
p |
|
Value
class htest
object, list
of
statistic |
test statistic |
parameter |
|
alternative |
alternative hypothesis of the test |
p.value |
p-value |
method |
name of the test |
data.name |
name of the tested time-series |
References
Tiao, G.C., Box, G.E.P. 1981. Modeling multiple times series with applications. Journal of the American Statistical Association 76, 802–816.
Examples
data(MDMforecasts)
ts <- MDMforecasts$ts
forecasts <- MDMforecasts$forecasts
l <- loss(realized=ts,evaluated=forecasts,loss.type="SE")
d <- d_t(l)
TB_AR_test(d=d,p=10)
Checks for a Lag in VMA Process with Tiao-Box Procedure.
Description
This function helps to find a lag in stationary VMA process with Tiao-Box procedure, i.e., the lag length beyond which we are willing to assume that the autocorrelation is essentially zero.
Usage
TB_MA(d,q.max)
Arguments
d |
|
q.max |
|
Details
The function searches for correlations smaller than -2n^{-0.5}
or higher than 2n^{-0.5}
, where n
is the lenght of the time-series.
Value
numeric
indicating the found lag length
References
Tiao, G.C., Box, G.E.P. 1981. Modeling multiple times series with applications. Journal of the American Statistical Association 76, 802–816.
Examples
data(MDMforecasts)
ts <- MDMforecasts$ts
forecasts <- MDMforecasts$forecasts
l <- loss(realized=ts,evaluated=forecasts,loss.type="SE")
d <- d_t(l)
TB_MA(d=d,q.max=10)
Computes Loss Differential.
Description
This function computes loss differential, i.e., differences between losses from k+1
-th and k
-th models.
Usage
d_t(e)
Arguments
e |
|
Value
matrix
of loss differentials
References
Mariano R.S., Preve, D., 2012. Statistical tests for multiple forecast comparison. Journal of Econometrics 169, 123–130.
Examples
data(MDMforecasts)
ts <- MDMforecasts$ts
forecasts <- MDMforecasts$forecasts
l <- loss(realized=ts,evaluated=forecasts,loss.type="SE")
d <- d_t(l)
Computes Loss Function.
Description
This function computes various loss functions for given realized values of time-series and a collection of forecasts.
Usage
loss(realized,evaluated,loss.type)
Arguments
realized |
|
evaluated |
|
loss.type |
method to compute the loss function, |
Value
matrix
with columns corresponding to time index and rows to different models
References
Hyndman, R.J., Koehler, A.B. 2006. Another look at measures of forecast accuracy. International Journal of Forecasting 22, 679–688.
Taylor, S. J., 2005. Asset Price Dynamics, Volatility, and Prediction, Princeton University Press.
Triacca, U., 2018. Comparing Predictive Accuracy of Two Forecasts, https://www.lem.sssup.it/phd/documents/Lesson19.pdf.
Examples
data(MDMforecasts)
ts <- MDMforecasts$ts
forecasts <- MDMforecasts$forecasts
l <- loss(realized=ts,evaluated=forecasts,loss.type="SE")
Sample Data from Crude Oil Price Forecasting.
Description
Forecasts obtained from various methods applied to crude oil price.
Usage
data(oilforecasts)
Format
oilforecasts
is matrix
object such that its rows correspond to forecasts from various methods, i.e.,
-
REALIZED
is the forecasted time-series, -
DMA.DOW
is the forecast from Dynamic Model Averaging with the dynamic Occam's window, -
BMA.DOW
is the forecast from Bayesian Model Averaging with the dynamic Occam's window, -
DMA.1V
is the forecast from Dynamic Model Averaging applied only to one-variable models, -
BMA.1V
is the forecast from Bayesian Model Averaging applied only to one-variable models, -
DMS.1V
is the forecast from Dynamic Model Selection applied only to one-variable models, -
BMS.1V
is the forecast from Bayesian Model Selection applied only to one-variable models, -
TVP
is the forecast from Time-Varying Parameters regression, -
LASSO
is the forecast from LASSO regression, -
RIDGE
is the forecast from RIDGE regression, -
DYN.EL.NET
is the forecast from the elastic net regression, with the elastic net mixing parameter changing with time index, -
LARS
is the forecast from the least-angle regression, -
B.LASSO
is the forecast from the Bayesian LASSO regression, -
B.RIDGE
is the forecast from the Bayesian RIDGE regression, -
ARIMA
is the forecast from the best ARIMA model according to AIC, -
NAIVE
is the naive forecast, i.e., the last observation is taken as a one-step ahead prediction, -
MA
is the moving average.
Details
The data were taken from Juvenal and Petrella (2015). They cover the period between 1971 and 2009, and are in quarterly freqency. Time-series with missing observations were excluded from the original data set, resulting finally in 127 explanatory variables, instead of 150 in the original data set. In particular, the excluded time-series are the ones which start date is after 1971. The dependent time-series is the average world price of oil taken in logarithmic differences. The independent time-series represent various stationarity tranformations of macroeconomic and financial variables of the G7 countries and from the oil market, global economic activity and various commodity prices. The details of the original data set are given in the paper by Juvenal and Petrella (2015). The forecasting with various models, based on this data set, was done by the author of this package, just to provide some more concrete example set of forecasts. The independent variables were taken in the first lags. The forgetting parameters in DMA/DMS models were set to 0.99, resulting in the effective rolling window size of 100. Therefore, such a window was taken for the moving average. LASSO and RIDGE (also in the Bayesian versions), the elastic net, the least-angle regression and ARIMA models were estimated in rolling windows of the size of 100 observations. First 100 observations were excluded, and oilforecasts
consists of the remaining last observations. The estimations were done with the following packages fDMA
, forecast
, glmnet
, lars
and monomvn
.
References
Drachal, K. 2017. fDMA: Dynamic Model Averaging and Dynamic Model Selection for continuous outcomes. https://CRAN.R-project.org/package=fDMA
Friedman, J., Hastie, T., Tibshirani, R. 2010. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33, 1–22.
Gramacy, R.B. 2017. monomvn: Estimation for Multivariate Normal and Student-t Data with Monotone Missingness. https://CRAN.R-project.org/package=monomvn
Hastie, T., Efron, B. 2013. lars: Least Angle Regression, Lasso and Forward Stagewise. https://CRAN.R-project.org/package=lars
Hyndman, R.J., Khandakar. Y. 2008. Automatic time series forecasting: the forecast package for R. Journal of Statistical Software 26, 1–22.
Juvenal, L., Petrella, I. 2015. Speculation in the oil market. Journal of Applied Econometrics 30, 612–649.
Examples
data(oilforecasts)
ts <- oilforecasts[1,]
forecasts <- oilforecasts[-1,]
l <- loss(realized=ts,evaluated=forecasts,loss.type="SE")
d <- d_t(l)
q <- TB_MA(d=d,q.max=4)
MDM.selection(realized=ts,evaluated=forecasts,q=q,alpha=0.1,statistic="Sc",loss.type="SE")
Prints MDM
Object.
Description
The function prints selected outcomes obtained from MDM.selection
.
Usage
## S3 method for class 'MDM'
print(x, ...)
Arguments
x |
an object of |
... |
not used |
Details
The function prints models with outstanding predictive ability, their mean loss function, statistics corresponding to their loss differentials (they are the number of the models less one), and orders these statistics. It also prints the p-value of the test and the number of eliminated models. If no models with outstanding predictive ability were found, the funtions prints such an information.
Examples
data(MDMforecasts)
ts <- MDMforecasts$ts
forecasts <- MDMforecasts$forecasts
m <- MDM.selection(realized=ts,evaluated=forecasts,q=10,alpha=0.1,statistic="S",loss.type="AE")
print(m)