Howto: Issue Reporting

[Grumble]

Tony hinted I could a write How To on Issue reporting as "Would you write one?" so I took the hint.

 

I don't think there is much technical novelty to add to the topic (but I will try), Tony said my issue reports were good but all I did is based on the beta guide and the forums here. Perhaps there is one thing, I want to write good issue reports and this is because I did my share of debugging in the past.

To understand why do you want to write a good issue report you only need to understand (a little) how issue fixing works.

From the software engineer point of view there are 4 steps in fixing an issue, these are Reproducing the symptoms, Identifying the bug, Analyzing it and then Coding the fix, "RIAC". However in the case of a released commercial software running in various uncontrolled environments by uncontrolled users it is rather: "RRRRRRRRRRRRRRRRRRRRIIIIIIIIIIIIIAAC".

It can easily take 10-20 times as much time to reproduce the bug as coding the fix.

The Reporter (beta tester) can have a big contribution here, making sure the issue is reproducible, providing clear instructions how to arrive to the error, providing log files and/or test saves, that is making sure that the usually expensive (or scarce or voluntary) software engineer resource spends his time on what only he can do, analyze and fix the code.

So you want to file a good issue report to cut out as many "R" and "I" as possible, ideally you want Tony just do a facepalm & "AC" and have the bug fixed by return post.

 

Ok, now that you are converted, take a look at technicalities:

• Read the the Beta Testers Read me First from Tony.

  http://harpgamer.com/harpforum/index.php?/topic/3574-beta-testers-read-me-first/

   

• Reproducing the error.
  • Iterative save.

    Iterative saving is your best friend here (for GE issues). Regardless of testing actually. But if you want to reproduce a perceived error you need a recent save. My quick launch toolbar h.bat is:

    call c:"\matrix games\cd2huce"

    winharp32 -S001800200

    -S001800200 creates saves every 00180 gametime seconds and rotates max 0200 pieces of numbered auto save file. The iterative save files are placed into the ITER_SAVE_GAMETIME subfolder under your save file folder. The iterative files are named as <your last manual save file name>.NNNN.hpX.

     

  • Reproduce.

    Ok, you have a recent save file. Can you reproduce the error again?

    No? then don't bother, Tony would just go G"RRRRRRRRRRRR...." too, if you can't reproduce others won't either.

    Yes? Great! You have cut out a bunch of "R"-s, go ahead.

     

  • Try it on a Stable version

    Especially if you are using the latest beta it is a good to test the issue with an earlier stable version. It might turn out that the issue was added with the recent beta codes. Let Tony know if earlier stable version does not throw the problem. (You are cutting "I"-s now.)

     

  • Record

    So you can reproduce the error, make sure others can too, record step by step what to do from loading the last save file to reproduce the error. ("R"-s again).

     

• Prepare the issue dump.
  • Collect to include in your issue report: description of the problem and how to reproduce it, version of GE/SE, custom DB if used, save files, possibly before and also after the problem occurs (for you).

     

  • IMPORTANT: Include the scenario file too (assuming a user scenario here). The format of the save file changes more frequently than the scenario format (at least in the past they did). Including the scenario file adds cross-version testability

     

  • Advanced dumping: create a test scenario (industrially cutting "I"-s )

    Every unit in a scenario adds (say) 10000 more lines of code to scan when trying to identify an issue. And that is per game second. You want to provide the smallest possible scenario which is able to reproduce the issue, with the bare minimum of units and with a save as close to the occurrence of the error as can be (without loosing the context).

    If you can fire up the SE and create scenario just with the offending units and can reproduce the error ... you can hear the buzzsaw going through the "I"-s!

     

• few more
  • One issue per report. Don't report multiple issues on the same record even if these are linked. It's just a way of loosing track of them. Keep 'em separate or copy&paste one per instance.

     

  • Remember show all - Ctrl-Alt-S

     

  • Loads of debug logging options are available, see the Beta Tester Manual (attached in the Read Me first).

     

  • Once there is a fix out, check back to the Issue Tracker and update the status of your report "Fix accepted by reporter" or extend if there is still an issue.

Remember, facepalm&AC!

 

/Grumble

[TonyE]

Excellent Grumble, thank you for the great write-up. Maybe I'll get the beta tester manual brought up to date as a result.

[JSF]

Tony,

 

is the WScenedit 32bit about to be tested this way, too? Or is it too early and You´ll someday give it free for testing and reporting?

 

I have started designing the second scenario with it and can report if that´s useful.

[TonyE]

Tony,

 

is the WScenedit 32bit about to be tested this way, too?

Yes, in fact you might say I'm purposefully not fixing issues in it so far unless they are reported (seeing what kind of commitment there is out there).

[JSF]

Ok, when I am back from the Canary Islands, I proceed with scenario editing and will report issues.