Abstract

This article investigates the differences between the rankings of funds produced by Morningstar’s risk-adjusted rating (RAR), and those generated by other risk-adjusted performance measures. The RAR appears to be the most widely used measure in the industry. Del Guercio and Tkac (2001) showed that Morningstar ratings have a considerable influence on U.S. investors’ behaviour towards fund selection. Meanwhile, Sharpe (1998) demonstrated that selecting funds within peer groups constructed by using the Morningstar rating as the only criterion was not efficient for building a multi-fund portfolio. Some authors also documented an age bias problem in Morningstar’s ratings: newer funds were more likely to receive five-star ratings than older funds (cf. Blume (1998) and Morey (2000)).

The present article analyses Morningstar ratings from a point of view that is different from that of previous studies. The authors investigate whether the funds would receive a different star rating if a risk-adjusted measure different from Morningstar’s RAR were used to perform the ranking. The following measures were used for the study: the CAPM, the Fama-French (1993) three factor model, and the return-based style analysis of Sharpe (1992). These three measures differ from the one used by Morningstar in that they rely on a theoretical background.

The analysis performed in this article is based on a subset of 1,730 domestic stock funds from the Morningstar Principia data set. The funds were selected on the following criteria: portfolio allocations to foreign stocks were less than 5%; no portfolio allocations to bonds; portfolio allocations to an "other asset class" that were less than 5%; and return history of sufficient length to allow for a Morningstar ranking. The returns used cover a period beginning in January 1992 and ending in December 2001. The article first gives a description of Morningstar’s ratings methodology. The present study is based on the methodology as it was before the modifications made in 2002.

Using the same methodology as Morningstar, but with different risk-adjusted measures, the authors performed a ranking of the funds. As in the Morningstar rating, three different period lengths were considered: 36 months, 60 months and 120 months, whenever the return series were available. The ratings were then attributed to the funds using the same rules as Morningstar. Survivorship bias was taken into account in the study. The results of the study show that the extent to which the choice of the risk measure impacts the ratings depends on the evaluation time frame. When the evaluation time frame is short, say three or five years, there is a strong correlation between the rating obtained with Morningstar's RAR and the rating obtained with the CAPM and Sharpe’s style analysis model, and a slightly lower correlation with the rating obtained from the Fama-French three factor model. However, when the evaluation time frame is extended to ten years, the correlation coefficients of the Morningstar rating with the others are distributed over a wide range, with the lowest value for the coefficient with the Fama-French three factor model rating.

To further characterise the ranking properties of the RAR, the authors calculated the empirical frequencies with which Morningstar star ratings were different from the alternative ratings. Approximately three out of four funds have the same star ratings regardless of whether the RAR or the excess return from the CAPM regression is used to rank funds. Conversely, the comparison with the Fama-French three factor model exhibits high empirical frequencies of rating inconsistency that increases over time.

The authors conclude that, overall, the ranking properties of the RAR are similar to those of the excess return from the CAPM regression, especially when the evaluation time frame is long, but that systematic differences exist between the star ratings produced by the RAR and the excess return estimated from the Fama-French (1993) three factor model. These results imply that the information provided by Morningstar’s rating will be of no use in evaluating long-term performance for an investor who models asset prices using the Fama-French three factor model, but will suit the needs of an investor using the CAPM for long-term performance evaluation. The authors also note that the selection of U.S. equity funds based on Morningstar star ratings only may produce a sub-optimal allocation across style categories for a multi-fund portfolio. For the period 1991 to 2001, this would have led to equity portfolios being over-allocated to growth stocks.

References

Blume M., “An Anatomy of Morningstar Ratings”, Financial Analysts Journal, vol. 54, 1998.

Del Guercio D., Tkac P., “Star Power: the Effect of Morningstar Ratings on Mutual Fund Flows”, Working Paper, University of Oregon, 2001.

Fama E., French K., “Common Risk Factors in the Return on Stocks and Bonds”, Journal of Financial Economics, vol. 33, 1993.

Morey M., “Mutual Fund Age and Morningstar Ratings”, Working Paper, Pace University, 2000.

Sharpe W., “Asset Allocation, Management Style and Performance Measurement”, Journal of Portfolio Management, vol. 18, 1992.

Sharpe W., “Morningstar’s Risk-Adjusted Ratings”, Financial Analysts Journal, vol. 54, 1998.