Site hosted by Angelfire.com: Build your free website today!

Sample of Using Tables for Layout

 AECT '99 Houston, TX

"Buying" your Grade? Creative Evaluation Strategies Using Authoring Software for Game Development

F.R. Koontz, Ph.D. ; Tami L. Schultz, A.B.D.

University of Toledo

In order to alleviate students' test anxiety, a Jeopardy-style game was developed and field tested for feedback and product revision. The results of the evaluation survey are shown here. The game was assessed both on an overall basis and on the basis of how well each major section performed. Specific items address both aesthetic and functional aspects of the game.

Rating scale:

1= lowest/worst rating;

5=highest/best rating

Item averages are given at the end of each statement. Topic averages are given at the beginning of each topic. Issues with light green backgrounds need first attention during next phase of development

Jeopardy! Evaluation Form

RESULTS:

General Program Issues

Avg: 4.49

Visual aspects

1

2

3

4

5

In general, screens are easy to read (uncluttered). avg = 4.64

1

2

3

4

5

In general, style of font is easy to read (uncluttered). avg = 4.64

1

2

3

4

5

In general, color combinations are easy on the eyes. avg = 4.55

1

2

3

4

5

In general, screens are appealing to look at. avg = 4.00

Avg: 4.64

Functional aspects

1

2

3

4

5

In general, buttons are located in an easy to find location on cards. avg = 4.82

1

2

3

4

5

In general, color coding aided with keeping track of where you were in the program. avg = 4.36

1

2

3

4

5

In general,the program screens were consistent in their layout from task to task. avg = 4.73

1

2

3

4

5

In general, the program took you to the type of information you expected when you clicked on a button. avg = 4.64

1

2

3

4

5

In general, you were not caught (stuck) in a loop of screens from which you could not escape. avg = 4.27

1

2

3

4

5

You were not required to know the way Hyperstudio works to operate the program. avg = 5.00

Specific Portions of the Program (Fill out as you work through the program)

Avg: 4.58

Title card

1

2

3

4

5

Color combination is appealing. avg = 4.50

1

2

3

4

5

Font used is appealing. avg = 4.25

1

2

3

4

5

Buttons work. avg = 4.75

1

2

3

4

5

Buttons take you to the type of information you expect. avg = 4.83

Suggestions for changes to title card:

  • small writing is somewhat choppy

Avg: 4.36

Direction cards

1

2

3

4

5

Color combination is appealing. avg = avg = 4.50

1

2

3

4

5

Font used is appealing. avg = 4.33

1

2

3

4

5

Buttons work. (Quit exits the program. To start the directions again, open the folder called “Jeopardy” in the temporary storage folder on the hard drive, then double click on the file called “Intro stack”.). avg = 4.55

1

2

3

4

5

Buttons take you to the type of information you expect. avg = 4.73

1

2

3

4

5

Directions were clear on how to play the game. avg = 3.92

1

2

3

4

5

Directions were clear on game requirements (number of questions required in each topic and dollar value level). avg = 3.83

1

2

3

4

5

Directions were clear on how the game would be scored. avg = 4.18

1

2

3

4

5

Directions were clear on how to quit the game. avg = 4.83

Was anything missing that you wanted to know? If yes, what?

Suggestions for changes to direction cards:

  • be able to return to directions from ANY point in the game
  • clean up typos
  • use icons in directions if possible
  • required too much scrolling--break up into more separate cards
  • score example was easier to understand than written directions

Avg: 4.33

Round status cards

1

2

3

4

5

Color combination is appealing. avg = 4.67

1

2

3

4

5

Font used is appealing. avg = 4.58

1

2

3

4

5

Buttons work. (Score button works, but information revealed is not functioning properly yet.) avg = 4.17

1

2

3

4

5

Buttons take you to the type of information you expect. avg = 4.27

1

2

3

4

5

It is clear that to select the topic, you have to click on the button at the bottom of the column. avg = 3.58

1

2

3

4

5

It is clear that attempted question values will be shown in red (this feature is presently not working properly. avg = 4.56

1

2

3

4

5

It is clear how to switch between round 1 and round 2 topics. avg = 4.67

1

2

3

4

5

The color coding on the topic buttons and the round number designation at the top of the screen is helpful for identifying which round’s status is showing. avg = 4.17

Suggestions for changes to round status cards:

  • add button to return to directions
  • change the font of the "Round 1 Status" title
  • click on $ amount rather than in front of it

Avg: 4.58

Topic Overview (light blue cards)

1

2

3

4

5

Color combination is appealing. avg = 4.33

1

2

3

4

5

Font used is appealing. avg = 4.50

1

2

3

4

5

Buttons work. avg = 4.50

1

2

3

4

5

Buttons take you to the type of information you expect. avg = 4.83

1

2

3

4

5

It is clear that to select the question, you have to click on the appropriate dollar value. avg = 4.58

1

2

3

4

5

It is clear that attempted question values will disappear after you click on them. avg = 4.58

1

2

3

4

5

It is clear how to check your topic score. avg = 4.75

Suggestions for changes to topic overview card:

  • visually cue the item that has been selected as the student's answer
  • some questions were not registered
  • put $ value on a button to keep it consistent
  • background color $ dollar value colors are too similar

Avg: 4.10

Individual question cards

1

2

3

4

5

Color combination is appealing. avg = 4.72

1

2

3

4

5

Font used is appealing. avg = 4.72

1

2

3

4

5

Buttons work. avg = 3.9

1

2

3

4

5

Buttons take you to the type of information you expect. avg = 3.8

1

2

3

4

5

It is clear that to answer the question, you have to click the tip of the cursor on the white circle in front of the answer. avg = 3.64

1

2

3

4

5

It is clear how many points you have attempted (this feature is currently not working. avg = 3.8

Suggestions for changes to individual question cards:

  • have circles in front of selected answer change in some way to indicate that particular answer was chosen
  • make sure the total dollar value the student has attempted to that point in the game is working before next round of testing

Avg: 4.46

Topic Score card (purple card with feedback)

1

2

3

4

5

Color combination is appealing. avg = 4.67

1

2

3

4

5

Font used is appealing. avg = 4.67

1

2

3

4

5

Feedback is motivating. avg = 4.50

1

2

3

4

5

Buttons (OOPS!) take you to the type of information you expect (returns you to the question you missed). avg = 4.50

1

2

3

4

5

It is clear that to go to another question, you have to click the button on the card. avg = 4.64

1

2

3

4

5

It is clear how many points you have attempted (this feature is currently not working.) avg = 3.8

Suggestions for changes to topic score card:

  • make link back to questions from all answers, regardless of whether they are right or wrong
  • correct answers were lost from the topic score card when question was returned to for correction

Overall Functionality

Results  

Criteria

yes

yes

no

Can the program be used by the student without assistance? (3 no, 9 yes)

yes

yes

no

Does the program provide the student with adequate instructions for its use? (1 no, 11 yes)

yes

yes

no

Can the program run without distracting the learner? (Color combinations, size of font, style of font, etc.) (all yes)

yes

yes

no

Is information presented in a logical order? (all yes)

yes

yes

no

Is the game generally appealing? (all yes)

yes

yes

no

Can instructions be easily bypassed for frequent players? (2 no, 10 yes)

yes

yes

no

Is it easy to find where to quit the game? (1 no, 11 yes)

no

yes

no

Can instructions be accessed easily? (9 no, 3 yes)

50/50

yes

no

Does the program respond as predicted to incorrect responses? (6 no, 6 yes)

yes

yes

no

Do the colors used motivate the student? (1 no, 11 yes)

Future Uses of this Game

Avg: 4.44

Overall Future Use

1

2

3

4

5

I would be willing to use this game to review for a test. avg = 4.75

1

2

3

4

5

I would have fun using this game to review for a test. avg = 4.83

1

2

3

4

5

I would be willing to use this game AS an alternative format for a test. avg = 4.08

1

2

3

4

5

I would have more fun using this game as a test than taking a test in the traditional paper and pencil method. avg = 4.08

Summary & Conclusions of Software Evaluation

Overall grade given to the game: (circle one)

A: 1

A-/B+: 8

B: 3

B-/C+

C

C-/D+

D

worse!

Provide an overall evaluation of the software. Briefly discuss the elements or events that are of particular value to you as a learner. Discuss the main shortcomings. What would you suggest be done to overcome the shortcomings?

Strengths:

Weaknesses:

Solutions for weaknesses:

Other general comments: