We have been experimenting with using Pixar's Braintrust idea to conduct critical project reviews.

Pixar uses a "Braintrust" to critically evaluate new movies in development. It is an intentionally critical panel of subject matter experts that find everything wrong with movies they have in development.

They never tell Directors (i.e. Movie Project Managers) how to fix problems they identify, since the “how” is the Director's job, but rather they identify problems that need to be fixed. The point is that it is designed to be critical because they want to find failures before they fail in production, and then to fix them during development.

This is the same idea behind the early warning elements of FTTM. Spot problems before they become major failures. In order for this work, an organization must create the conditions for it to work.

But you have to create an environment where failure is accpted as an incremental part of the learning process. These learning environments are rare in corporate cultures today, especially in technology development.

So we asked, "What about a Braintrust to scrub projects?"The idea would be to prevent failures before they fail. A critical panel who spots problems and provides critical feedback, which would then be converted by core teams into constructive improvements to their projects; be it in product design, definition, organization, decision-making, etc. We are experimenting with this idea and will report back our findings over time.

Here’s an extract from Ed Catmull's book that sums up the learning mindset (the book is well worth the read):

“But it’s not what makes Pixar special. What makes Pixar special is that we acknowledge we will always have problems, many of them hidden from our view; that we work hard to uncover these problems, even if doing so means making ourselves uncomfortable; and that, when we come across a problem, we marshal all of our energies to solve it. This, more than any elaborate party or turreted workstation, is why I love coming to work in the morning. It is what motivates me and gives me a definite sense of mission.”

Excerpt From: Ed Catmull with Amy Wallace. “Creativity, Inc.” iBooks.