EDHEC-Risk - Rating the Ratings

EDHEC-Risk Concept Industry Analysis Featured Analysis Latest EDHEC-Risk Surveys Research News Research Papers Books Features Interviews Indexes and Benchmarking EDHEC-Risk Efficient Equity Indices FTSE EDHEC-Risk ERAFP SRI Index Equity Index Research EDHEC-Risk Alternative Indexes Hedge Fund Index Research EDHEC-Risk IEIF Commercial Property Indices Amundi ETF "Core-Satellite and ETF Investment" Research Chair Solvency II Benchmarks Style and Performance Analysis Hedge Fund Performance EuroPerformance/EDHEC-Risk Institute Style Ratings Performance Measurement for Traditional Investment Asset Allocation and Alternative Diversification Real Assets Newedge "Advanced Modelling for Alternative Investments" Research Chair CME Group "Exploring the Commodity Futures Risk Premium: Implications for Asset Allocation and Regulation" Strategic Research Project SGCIB "Structured Equity Investment Strategies for Long-Term Asian Investors" Strategic Research Project Asset Allocation and Derivative Instruments Structured Forms of Investment Strategies FBF "Structured Products and Derivatives" Research Chair Eurex "The Benefits of Volatility Derivatives in Equity Portfolio Management" Strategic Research Project ALM and Asset Management AXA Investment Managers "Regulation and Institutional Investment" Research Chair BNP Paribas Investment Partners "ALM and Institutional Investment Management" Research Chair Deutsche Bank "Asset-Liability Management Techniques for Sovereign Wealth Fund Management" Research Chair Ontario Teachers' Pension Plan "Advanced Investment Solutions for Liability Hedging for Inflation Risk" Research Chair Rothschild & Cie "The Case for Inflation-Linked Corporate Bonds: Issuers' and Investors' Perspectives" Research Chair Russell Investments "Solvency II Benchmarks" Research Chair Operational Risks and Performance Best Execution: MiFID and TCA Mitigating Hedge Funds Operational Risks CACEIS "Risk and Regulation in the European Fund Management Industry" Research Chair EDHEC-Risk Publications Reports, Studies, Surveys and Position Papers Academic Publications All EDHEC-Risk Publications Events Events organised by EDHEC-Risk Institute Analysing Sovereign Risk for Portfolio Management Decisions Seminar, London, 12-13 June, 2012 CFA Institute/EDHEC-Risk Institute Advances in Asset Allocation Seminar, New York, 12-14 June, 2012 Advanced Commodity Investment Seminar, London, 19-20 June, 2012, New York, 16-17 July, 2012 New Frontiers in Equity Investing Seminar, Boston, 26-27 June, 2012 Events involving EDHEC-Risk Institute's participation EDHEC-Risk Institute Presentation Research Programmes Research Chairs and Strategic Research Projects Partnership IPE EDHEC-Risk Institute Research Insights International Advisory Board Team EDHEC-Risk News EDHEC-Risk Newsletter EDHEC-Risk Press Releases EDHEC-Risk in the Press Careers EDHEC Business School EDHEC-Risk Executive Education EDHEC-Risk Institute PhD in Finance EDHEC-Risk Institute Executive MSc in Risk and Investment Management Investment Management Seminars Contact Us Contact Us

Rating the Ratings

In 2002, EuroPerformance, which is the leading French firm for the dissemination of mutual fund data, approached EDHEC-Risk Institute to consider the implementation of a value-added offering in the area of external analysis of the performance and risks of European investment funds.

The method followed to implement the design of these new ratings was to carry out a thorough study of the insufficiencies of the existing rating methods in order to correct them by relying on the state-of-the-art in portfolio risk and performance measurement in a business context.

The Rating the Ratings document constitutes a detailed summary of the critical study and puts into perspective the responses given to the inadequacies of the existing ratings by the EuroPerformance/EDHEC-Risk Institute Style Ratings.

The ratings compare funds that are not comparable

The analysis of the main existing ratings carried out by EDHEC-Risk Institute shows that for numerous categories, notably the categories corresponding to equity funds invested in large-cap stocks and internationally diversified funds, the rankings contain a mix of investment vehicles that present totally different risks and as such are not comparable. The style exposure averages in the “equities” category are not representative, since the mean dispersion (standard deviation) of the weights of each style is often greater than their average.

The performance-related ranking methods based on the Information Ratio implemented by Lipper and S&P exacerbate the fragility of the ratings linked to the poor definition of categories, because the ranking is extremely sensitive to the choice of index, which is included in the calculation of both outperformance and relative volatility (tracking error).

More globally, all of the ratings are based on rankings that are related to categories, which makes them particularly fragile and not very robust, since they depend on the prior definition of the categories. The stars attributed in each category are not comparable and do not have the same value. Depending on the breadth of the category to which they belong, the same funds can find themselves being attributed different levels of stars.

The style of the funds is not genuinely taken into account

Even though academic research work has shown that the choice of style (Growth, Value, Small Cap or Large Cap) was a determining factor in a portfolio’s risk and performance over a long period, this notion is either poorly taken into account or not taken into account in the ratings.

As a result, none of the major international rating agencies are capable of distinguishing clearly between value funds and growth funds in the European market. Moreover, when an agency attempts to do this, the integration of the style is not based on any serious analysis. Almost 70% of the funds ranked as Large Cap in the Morningstar Global Equity or Equity Euroland Large Cap categories actually have a majority exposure to the Small Cap style.

Alphas missing from performance measurement and scores that depend on the chance of financial markets

Furthermore, these approaches do not lead to each fund being attributed a benchmark that is representative of the risks taken. The fund’s ranking is not therefore produced on the basis of its real outperformance (alpha), but depends on the overall performance which itself is conditioned strongly by whether the prevailing situation is favourable for the styles to which the mutual funds are exposed. With the exception of the scores attributed by Aptimum, the authors of the study estimate that, for the vast majority of rating categories, the scores attributed do not depend on either the manager’s stock picking or tactical allocation capability, but on their exposure over a long period to one or more styles, without it being possible to say whether this selection of risks over the long period is due to skill or luck.

This inability to distinguish between performance and outperformance makes any use of ratings as a tool for selecting managers or funds illusory. For the most part, the best scores obtained come from the evolution of the risk premiums of the styles to which the funds are exposed. Financial theory postulates that these follow a random walk or, at least, cannot be determined by only observing past values.

Potential extreme loss not taken into account; the lessons from the stock market crashes have still not been learnt

The study also shows that the calculation of the extreme loss potential (VaR) to which the funds are exposed is either not performed (Morningstar, S&P and Aptimum) or not taken into account (Lipper) in the fund’s overall score, even though funds are increasingly turning to derivative instruments, and are invested in assets that present non-normal distributions, which makes the use of volatility as the only risk measure illusory.

Performance persistence measurement is inadequate

Finally, performance persistence measurement is not given enough attention. Most rating agencies do not provide a satisfactory solution, either because of a lack or inadequacy of the measure (S&P, Morningstar and Aptimum), or because it is not integrated into the overall score (Lipper).

New ratings to respond to the inadequacies of the current ratings

On the basis of the critiques and the solutions proposed by modern portfolio theory, EuroPerformance and EDHEC-Risk Institute have proposed a method that breaks free from the criticism described above:

It is an absolute rating that only allows a fund-specific benchmark to be calculated for each of the funds. The attribution of stars no longer depends on a category.
The outperformance (alpha) is the principal criterion for attributing stars and alpha is a magnitude that does not depend on the prevailing situation and thus on the chance of the financial markets, but only on the manager’s skill, the persistence of which can be measured.
The fund’s extreme loss potential (VaR) is measured and integrated into the overall rating process.
The measurement of the persistence of outperformance is subjected to a thorough quantitative analysis and is also integrated into the overall rating process.

Evaluation of the different ratings

Attachments

View full version of the "Rating the Ratings" study

FTSE EDHEC-Risk Efficient Indexes: April 2012

United States	0.21%
United Kingdom	-0.91%
Eurobloc	-3.13%
Developed Europe	-1.42%
Dev. Europe ex. UK	-2.49%
Japan	-5.29%
Dev. Asia ex. Jap.	-0.17%
Asia-Pac. ex. Jap.	-0.07%
Asia-Pacific	-0.89%
Developed	-0.41%
Emerging	-0.95%
All World ex. US	-1.02%
All World ex. UK	-0.57%
All World	-0.47%

EDHEC-Risk Alternative Indexes: Apr 2012 (Estimates)

Conv. Arb.	-0.23%
CTA Global	-0.01%
Dist. Sec.	-0.11%
Emg. Mkts	-0.45%
Eq. Mkt Neut.	-0.08%
Event Driven	-0.14%
Fix. Inc. Arb.	0.50%
Global Macro	-0.49%
L/S Equity	-0.65%
Merger Arb.	-0.13%
Rel. Value	-0.23%
Short Selling	1.02%
FoF	-0.27%

EDHEC-Risk IEIF Commercial Property: April 2012

Price (FR)	0.64%
Total Return (FR)	1.90%