As you may already know, we would like to implement several changes to improve
our current project initiation and development process.  The purpose of this 
memo is to document the proposed changes/additions so that we can move forward
with improved communication, efficiency, and most of all, even better 
products.

Please review and comment on any additional improvements you can offer.

As a starting reference, we summarized some of the flaws in our present 
system, many of which were brought up in our recent brainstorm sessions:

   Lack of overall company direction/strategy.

   Low awareness of the market conditions, product requirements. 
   (Communication not there).

   Projects inititated prematurely in some cases.
  
   Inconsistent presentation of game proposals at initiation.

   Need for more player feedback throughout the development.
 
   Poor definition of the development process with some teams taking
   on their own approach. (e.g., how they interface with support groups).

   Teams working in isolation too much.

   Difficult to provide input that will be heard unless that individual 
   qualifies to have an opinion by playing the game frequently and being 
   involved early in development.

   Ineffective product reviews (vague objectives, too premature, and too much 
   of a dreaded event for the team).

   Difficult to evaluate product potential when field tests plagued with bugs,
   tuning needs.  Need time for more testing, distributor sampling. 

   Need more accurate forecasting methods for sales volumes.    

We believe many of these problems can be resolved by adapting the current 
development process.  Other issues can be addressed by incorporating some 
of the ideas that we mentioned throughout the brainstorming process. 

John Ray is documenting many ideas on how we can improve our overall 
effectiveness. 

Attendees:  Marketing, sales, engineering management, project teams.

Purpose:  This meeting will be held approximately every quarter to help 
engineering select a game concept or modify their current projects.

Marketing will provide information on competition, players, game design 
trends, etc.  Ideally, marketing and engineering will provide a list of 
the type of games they feel are appropriate to initiate, based on the  
market and the current projects already in development.

A summary of the meeting and a video tape will be made available. 

We all feel it would be much more productive if we break up the initiation 
process to include meetings prior to a formal initiation packet. These 
additional meetings will serve as "checkpoints" to insure that the proposed 
project is in line with current company needs and objectives.

PRODUCT STRATEGY MEETING:  

   In addition to the marketing input meetings, each team that is up for 
   a new project should have an informal discussion with marketing and 
   engineering management to review current product direction.  Any rough 
   concepts that are being considered could be discussed briefly to get 
   initial reactions from those present. 
                                 
PITCH SESSIONS: 

   Attendees:  Marketing, engineering management, project teams, and any 
   employee who wants to pitch their concept.

   Purpose: Individuals or groups will have an opportunity to describe their 
   game idea to all teams who are interested.  Marketing will screen all game 
   concepts before they are further developed for a pitch session.  A "product
   champion" from the marketing group will be assigned to provide direction 
   and support.  Pitch sessions should be videotaped for those unable to 
   attend.

   Details of the format/incentive for pitch sessions need to be worked out.  

   Tom Steiner has provided a handout on a proposed format and is working 
   on a videotape on "How to Pitch", which  will be available for reference.


CONCEPT APPROVAL MEETING:
 
   Attendees: Marketing, engineering management, project team.

   Purpose:   Once the team has selected a project idea, it should be
   "concept" approved by engineering management and marketing before going to 
   a full initiation.  In the concept approval process, it is the core game 
   play that will be critically evaluated.  Feedback is given at this early 
   stage to avoid wasted efforts in preparing for a full-blown initiation.  
   The marketability issues will also be evaluated (price point, product need,
   sales potential).  Once the idea is approved, a marketing product manager 
   will be assigned to the team and a project number will also be given.  
   If the concept is not approved, the team selects another concept and holds 
   another concept approval meeting.

   Preparation Details:  The project team should work on the concept for no 
   more than two weeks before holding the concept approval meeting.  The 
   following items should be covered: 

        Theme/game play. The basic, lowest level, moment-to-moment action. 
        What does the player  actually do?  How do the controls affect the 
	action?  How do opponents/playfield elements respond to what the 
	player is doing?
 
        Simple sketches illustrating the look of the game?

        Controller types and actions.

        Type of hardware used. Unique research or engineering tasks that would
        have to be performed (new hardware, new playfield software, marketing
        studies, etc.)

INITIATION MEETING: 

	Attendees:...  Marketing, engineering management, project team

	Purpose:  The next stage would be the initiation meeting which should 
	be a work session where specific game design issues are evaluated and 
	input is given for final direction of the project. 

	Specific items that need to be defined in the initiation packet:
 
     1.  Theme/storyline 
     2.  Game objectives/goals 
     3.  Controllers and function
     4.  Wave progression
     5.  Action flow (example of moment-to-moment action in one wave/level).
     6.  Audio requirements (use of speech, style of music)
     7.  Animation/graphic treatment, playfield description
     9.  Player Adversaries
    10.  Cabinet direction
    11.  Hardware considerations  
    12.  Cost estimates (set limits on cost, EPROM count, etc.)
    13.  Development schedule (include goals for first review)
    14.  Marketing issues/considerations 
    15.  Storyboards of various phases
    16.  Picture of sample screen if video is novel (RAD tool as an example)
    17.  Team personnel. We would also like a section to include "marketing 
         issues" which would cover the product logic (targeted 
         locations/players), critical elements of the game to maximize its 
         success, possibility for multiple product configurations, price 
	 points, timing in the marketplace, etc.  The assigned product manager
	 can provide this section of the packet after discussing with the 
	 project leader. Storyboards are a critical tool for the initiation 
	 meeting.  Rather than concentrate on a few, highly detailed, color 
	 boards, it would be more effective to have numerous quick sketches 
	 which depict the type of player interaction that can be expected.   
	 If the attendees agree that the game is well-defined and on the right
	 track, then the game is initiated.  (If there are problems that still
	 need to be addressed, then the team can make appropriate 
	 changes/additions and schedule another initiation meeting)

When and why we review games needs to be more flexible, although ideally, 
the current number is about right (4 before field test).    

If there are specific concerns raised by the team or marketing/management, 
then this should be addressed at an informal review.  If there are changes 
in the market conditions which would impact the project then a review could 
be scheduled as well.  In any case, reviews should be mutually determined by 
the project leader and the product manager, with an agreement on the goals 
for that review.  The product manager will be informally evaluating assigned
projects more frequently throughout the development process.  By doing this 
we hope to eliminate any surprises at reviews  (e.g., major problems with the 
game concept or playability heard for the first time.)

PRIOR TO REVIEWS: 

   In order to get more thorough and constructive feedback during the review 
process, the project team should have the game playable for the 24 hours 
preceding the review. This should allow everyone time to formulate comments 
and feedback.  The P/L should send out notices of the review date/time.  
A day before the review, all attendees should receive a list which documents 
the agenda items for discussion and the goals of the review.  A brief 
description of the current state of the game as well as changes since the last
review should be included so attendees understand what they should be 
evaluating.  

DURING REVIEWS: 

   Consistent review of all agenda items should be done in a structured way to
make sure it is fully discussed.  The team should critique their game first, 
discussing known strengths and weaknesses.  If the team discusses what they 
plan to do to address known weaknesses, then they can avoid hearing redundant 
negative comments. Another thought is to have the P/L and product manager 
review game play. Especifics much like we do in a focus group.  

   We should objectively revisit the goals of the project as set forth at 
initiation what are the strengths of this game and how can we capitalize 
on it.    

   Directional input and action items should be verbally summarized at the end
of the review so that all attendees have a clear understanding of the outcome.
 
POST REVIEWS:  
   
   The project leader should document the results of the review and include a 
list of all action items and subsequent changes in scheduling if impacted.  
Revision of project goals should also be specified if necessary.

   What we should avoid is amibiguity in the direction we are taking (e.g., 
concern raised about the lack of a goal in the game (we will "look into it.")

   As a rough guideline, we should continue to schedule reviews at certain 
stages of development, much like in the past:

   First review objectives typically include evaluation of the first-pass 
   look and feel of the game (rough graphics and player control) and hardware
   demonstration, if new or unique application.

   Second review objectives are to demonstrate a near-complete wave or level, 
   with concentration on the moment-to-moment action.  Player comprehension 
   is generally an issue at this stage.  The focus group schedule can be 
   discussed, depending on the confidence level of the project.  

   By the third review, a feel for the game progression should be evident 
   as multiple waves are done.  Adversary interaction, level of 
   decision-making, player goals, and measurement of performance are some of 
   the more critical elements that should be in the game

   The fourth review is usually after focus group and before a field test.
   If a focus group has been conducted, then we revisit any directional 
   changes at this review.  Game elements that need to be included for field
   test are also discussed at this review.  

   Post field test reviews vary in number and frequency, depending on what
   we learn from player reaction, game statistics, etc.

   We have relied on focus groups (30 players) and field surveys (20-40 
   players) for our external evaluation of product.  Where needed, we can 
   consider doing earlier focus testing, and perhaps more in-depth 
   interviewing of players who are recruited from field surveys.

   When necessary we can call in playtesters (either internal or external) to 
   provide additional in put in the development process. 
 
FOCUS GROUPS:  

   The purpose is to determine perceived strengths and weaknesses of a 
product, comprehension issues, and a sense of who the product may appeal to 
most.  When we have specific concerns about a product we will consider earlier
focus groups.  As an example, if controls are a major directional issue, then 
we can conduct a controller test.  Other examples include theme or unique game
format.

   In some cases, it may be appropriate to skip a focus group altogether 
depending on the confidence level of the game.  

FIELD TESTING:  

   The purpose of the field test varies in stages.  As with the entire 
development process, the time required for testing depends on what we learn at
each critical stage.  The product manager publishes a field test schedule 
which indicates the number of units required and timing for prototypes and 
pre-production units.  Specific objectives for each field test should be 
defined once discussed with the team. The objective of the first field test is
to measure initial earnings potential in a typical arcade.
 
   Player feedback is gathered from the field to again look at strengths and 
weaknesses of the game, and to profile the player base.
      
   The second field test is started once the issues from the first test are 
addressed to some extent.  This test is also intended to be a longevity test 
where earnings and the learning curve are evaluated for a period of 6-8 weeks.

   Next test units are used to measure street location potential and also to 
test various coinage options or difficulty variables.

   The fourth round of testing is to evaluate earnings/appeal out of the area.
Whenever possible, we will push to have samples available at least four weeks 
prior to finished goods for distributor testing.  These are typically 
pre-production units which ship with a tuned program, and may need to be 
updated to a production version at a later date. 

   Throughout the testing process, game tuning is an ongoing objective, as it 
is a dynamic process up through production.

   Near the conclusion of the testing process the product manager will provide
a "sales volume forecast" which will be based on the collective field test 
results to date, the current market conditions, the product price, and the 
production timing.

   We have also reinstituted the tradition of having RIP meetings for every 
project with the intent of increasing our experience base. Final product 
evaluation will be discussed, as well as how we could have improved any 
part of the development process. This meeting should be scheduled by the 
product manager and project leader, and would be most effective if it occurs 
after we have objective field feedback from production shipments.


DISTRIBUTION:  R. DAWE
	       B. FLANAGAN
	       M. HALLY
	       D. HARPER
	       P. KWINN
	       E. LOGG
	       G. LICHAC
	       R. OWENS
	       M. PIERCE
	       D. RALSTON
	       J. SALWITZ
	       G. STARK
	       K. TURNER
	       S. SOITA

COPIES:        L. BENZLER
	       S. COMSTOCK
	       C. DOWNEND
	       B. FULLER
	       M. FUJIHARA
	       J. MOMODA
	       R. MOORE
	       R. MONCRIEF
	       H. NAKAJIMA
	       L. RAINS
	       J. RAY
	       P. TAKAICHI
	       D. VANELDEREN


This looks like a memo that was distributed to the Atari project teams. It
did not have a valid header in the original document, so I created one
with the same information in the memo's header.