Micro‑Benchmarking ‑ Measure as You Go

While designing or coding, you may be aware of different ways of accomplishing the same functionality in Jade. It can be beneficial to do a performance test to decide between them at the time, particularly if the same functionality will be used many times. Simple construct choices, in particular, benefit from measuring as you go, as they are easier to test.

Micro‑benchmarking refers to a simple test comparing two constructs, to see which is faster. These tests are frequently run in Workspace or JadeScript methods. The following is an example JadeScript method that is useful for comparing constructs.

jblPfm() updating;
vars
    oldWay : Boolean;
    dts,dte : Decimal[19];
    s,t,u,v : String;
    i,j,reps : Integer;
begin
    write CrLf & "Starting...";
    reps := 10000;  // adjust this to get a run length of 1 to 10 seconds
    oldWay := true;
    dts := app.relativeMachineTime;
    foreach i in 1 to reps do
        if oldWay then
            s := app.actualTimeServer.String;
        else
            s := app.actualTimeAppServer.String;
        endif;
    endforeach;
    dte := app.relativeMachineTime;
    write "<" & s & ">" & "<" & t & ">";  // to verify the functional results
    write ((dte‑dts)/reps).roundedTo(6).String & " ms/rep";
epilog
    write "Finished";
end;

When performing micro‑benchmarks, there are a few things to watch out for.

• If the test involves reading objects from the database, be aware that the objects will likely be in cache the second time the test is run. It is all too easy to test something the old way, make a change, run it again, see a big improvement, and assume it was the change that made the difference.

  It is possible that the primary difference was the fact that objects were cached the second time around ‑ in the node's cache or the database server's disk cache.

  If you need a reliable comparison of tests involving database IO, you will need to restart the database server between tests, to clear the database server's disk cache. This may be a big inconvenience, but it is essential for tests involving IO.

• If you measure your optimizations in a hot cache scenario (by not restarting), small differences in CPU can seem significant, when in real life they would be insignificant compared to the IO that is being done.

• If you are analyzing results using the Jade Monitor, be aware that running a Workspace or JadeScript method involves starting and stopping a Jade application, which may have a significant impact on the counters you are examining.

  Run an empty JadeScript method and analyze the activity involved, to see what the overhead is. The overheads can be especially large if the test uses classes that no other processes in the node are using, which is common on development systems. Unless the classes are already in use or Production mode is being used, executing a JadeScript method will incur the overhead of opening the classes. In addition, when the JadeScript method finishes, the classes will be closed (unless used elsewhere or Production mode is on), and all instances of those classes will be removed from the node's cache. This also applies to Workspaces. This can lead to big inconsistencies between runs; for example, if another user has the classes open for some runs but not other runs.

  As an alternative, you may want to use a simple form that allows you to run a test by clicking a button and displaying the results in a large text box display. This way you can avoid the overheads of starting and stopping an application, and the potential variability of opening and closing classes.

  If you do use a JadeScript or Workspace method, be sure to measure the time internally (as in the previous code example), and create and delete any work transients and so on outside the timed section – unless that is what you are benchmarking, of course!