EDHEC-Risk Concept Industry Analysis Featured Analysis Latest EDHEC Surveys Research News Research Papers Books Features Interviews Indexes and Benchmarking EDHEC Alternative Indexes EDHEC IEIF Commercial Property Indices Hedge Fund Indices Literature EDHEC's Position on the Eligibility of Hedge Fund Indices for UCITS Assessing the Quality of Stock Market Indices EDHEC European ETF Survey Core-Satellite Investing CASAM "Core-Satellite and ETF Investment" Research Chair Style and Performance Analysis Hedge Fund Performance EuroPerformance-EDHEC Style Ratings Alpha League Table IPE-EDHEC Institutional Asset Management Awards (IAMA) Rating the Ratings Performance Measurement for Traditional Investment Asset Allocation and Alternative Diversification EDHEC European Alternative Diversification Practices Survey Hedge Fund Style Allocation EDHEC Funds of Hedge Funds Reporting Survey The Amaranth Case The Hedge Fund Debate Core-Satellite Investing Morgan Stanley Investment Management "Financial Engineering and Global Alternative Portfolios for Institutional Investors" Research Chair Newedge "Advanced Modelling for Alternative Investments" Research Chair Asset Allocation and Derivative Instruments Structured Forms of Investment Strategies FBF "Structured Products and Derivatives" Research Chair Use of Derivatives in Asset Management ALM and Asset Management Solvency II Impact of IFRS & Solvency II on ALM & AM in Insurance Companies Managing Pension Assets Benefits of Hedge Funds in ALM ALM Decisions in Private Banking AXA Investment Managers "Regulation and Institutional Investment" Research Chair BNP Paribas Investment Partners "ALM and Institutional Investment Management" Research Chair ORTEC Finance "Private Asset-Liability Management" Research Chair Deutsche Bank "Asset-Liability Management Techniques for Sovereign Wealth Fund Management" Research Chair UFG "Dynamic Allocation Models and New Forms of Target-Date Funds for Private and Institutional Clients" Research Chair Best Execution and Operational Performance MiFID TCA in Europe: Current & Best Practices Mitigating Hedge Funds Operational Risks CACEIS, NYSE Euronext and SunGard "MiFID & Best Execution" Research Chair Events Events organised by EDHEC CAIA Level II Review Seminar, Paris, 1-3 July, 2009 Events involving EDHEC's participation EDHEC Publications Reports, Surveys and Position Papers Academic Publications All EDHEC Publications Investment Management Review Editorial Policy Subscriptions EDHEC Risk and Asset Management Research Centre Presentation Research Programmes Research Chairs International Advisory Board Partners Team EDHEC-Risk News Press Releases EDHEC in the Press Careers EDHEC Business School EDHEC Asset Management Education EDHEC PhD in Finance Asset Management Seminars CAIA Preparation Contact Us Contact Us
Performance and Style Analysis
Rating the Ratings

In 2002, EuroPerformance, which is the leading French firm for the dissemination of mutual fund data, approached the Edhec Risk and Asset Management Research centre to consider the implementation of a value-added offering in the area of external analysis of the performance and risks of European investment funds.

The method followed to implement the design of these new ratings was to carry out a thorough study of the insufficiencies of the existing rating methods in order to correct them by relying on the state-of-the-art in portfolio risk and performance measurement in a business context.

The Rating the Ratings document constitutes a detailed summary of the critical study and puts into perspective the responses given to the inadequacies of the existing ratings by the EuroPerformance-Edhec Style Ratings.

The ratings compare funds that are not comparable
The analysis of the main existing ratings carried out by Edhec shows that for numerous categories, notably the categories corresponding to equity funds invested in large-cap stocks and internationally diversified funds, the rankings contain a mix of investment vehicles that present totally different risks and as such are not comparable. The style exposure averages in the “equities” category are not representative, since the mean dispersion (standard deviation) of the weights of each style is often greater than their average.

The performance-related ranking methods based on the Information Ratio implemented by Lipper and S&P exacerbate the fragility of the ratings linked to the poor definition of categories, because the ranking is extremely sensitive to the choice of index, which is included in the calculation of both outperformance and relative volatility (tracking error).

More globally, all of the ratings are based on rankings that are related to categories, which makes them particularly fragile and not very robust, since they depend on the prior definition of the categories. The stars attributed in each category are not comparable and do not have the same value. Depending on the breadth of the category to which they belong, the same funds can find themselves being attributed different levels of stars.

The style of the funds is not genuinely taken into account
Even though academic research work has shown that the choice of style (Growth, Value, Small Cap or Large Cap) was a determining factor in a portfolio’s risk and performance over a long period, this notion is either poorly taken into account or not taken into account in the ratings.

As a result, none of the major international rating agencies are capable of distinguishing clearly between value funds and growth funds in the European market. Moreover, when an agency attempts to do this, the integration of the style is not based on any serious analysis. Almost 70% of the funds ranked as Large Cap in the Morningstar Global Equity or Equity Euroland Large Cap categories actually have a majority exposure to the Small Cap style.

Alphas missing from performance measurement and scores that depend on the chance of financial markets
Furthermore, these approaches do not lead to each fund being attributed a benchmark that is representative of the risks taken. The fund’s ranking is not therefore produced on the basis of its real outperformance (alpha), but depends on the overall performance which itself is conditioned strongly by whether the prevailing situation is favourable for the styles to which the mutual funds are exposed. With the exception of the scores attributed by Aptimum, the authors of the study estimate that, for the vast majority of rating categories, the scores attributed do not depend on either the manager’s stock picking or tactical allocation capability, but on their exposure over a long period to one or more styles, without it being possible to say whether this selection of risks over the long period is due to skill or luck.

This inability to distinguish between performance and outperformance makes any use of ratings as a tool for selecting managers or funds illusory. For the most part, the best scores obtained come from the evolution of the risk premiums of the styles to which the funds are exposed. Financial theory postulates that these follow a random walk or, at least, cannot be determined by only observing past values.

Potential extreme loss not taken into account; the lessons from the stock market crashes have still not been learnt
The study also shows that the calculation of the extreme loss potential (VaR) to which the funds are exposed is either not performed (Morningstar, S&P and Aptimum) or not taken into account (Lipper) in the fund’s overall score, even though funds are increasingly turning to derivative instruments, and are invested in assets that present non-normal distributions, which makes the use of volatility as the only risk measure illusory.

Performance persistence measurement is inadequate
Finally, performance persistence measurement is not given enough attention. Most rating agencies do not provide a satisfactory solution, either because of a lack or inadequacy of the measure (S&P, Morningstar and Aptimum), or because it is not integrated into the overall score (Lipper).

New ratings to respond to the inadequacies of the current ratings
On the basis of the critiques and the solutions proposed by modern portfolio theory, EuroPerformance and Edhec have proposed a method that breaks free from the criticism described above:
  • It is an absolute rating that only allows a fund-specific benchmark to be calculated for each of the funds. The attribution of stars no longer depends on a category.

  • The outperformance (alpha) is the principal criterion for attributing stars and alpha is a magnitude that does not depend on the prevailing situation and thus on the chance of the financial markets, but only on the manager’s skill, the persistence of which can be measured.

  • The fund’s extreme loss potential (VaR) is measured and integrated into the overall rating process.

  • The measurement of the persistence of outperformance is subjected to a thorough quantitative analysis and is also integrated into the overall rating process.
Evaluation of the different ratings
Attachments

View full version of the "Rating the Ratings" study