Charlwood Integration Test Suite

Summary of Test Results

Kevin Charlwood's 2008 article Integration on Computer Algebra Systems included 50 indefinite integration problems involving a variety of elementary functions. This chart summarizes the performance of Rubi and several computer algebra systems on these problems.

The following table shows the grade given each system on each of the 50 problems in the Charlwood Test Suite:

Problem # Rubi 4.8 Mathematica 10 Maple 18 FriCAS 1.2.1 Maxima 5.28 Derive 6.1
1 2 2 1 2 2 2
2 2 2 2 2 2 2
3 0 1 1 2 -1 0
4 1 1 1 1 0 1
5 1 1 1 0 0 0
6 2 0 2 2 0 1
7 2 1 2 2 2 2
8 2 0 1 0 0 0
9 2 1 0 2 2 2
10 2 1 2 2 0 0
11 2 2 0 2 0 2
12 1 1 1 2 0 0
13 1 1 0 2 0 0
14 2 2 0 1 0 2
15 2 1 0 2 0 2
16 2 2 0 1 2 2
17 2 1 0 1 0 2
18 2 2 1 2 2 2
19 2 2 2 2 2 2
20 2 2 0 2 2 2
21 2 1 0 1 0 0
22 2 1 0 2 0 1
23 2 2 0 2 0 2
24 2 2 2 2 2 2
25 2 2 0 1 2 2
26 2 2 0 1 0 2
27 2 2 2 1 2 2
28 2 2 0 1 2 2
29 2 2 0 2 0 2
30 2 2 0 2 0 2
31 2 2 0 1 2 2
32 2 2 2 2 2 2
33 2 2 1 1 2 2
34 2 2 2 2 2 2
35 2 2 0 2 0 2
36 2 2 2 1 2 2
37 2 1 2 1 2 2
38 2 1 1 2 0 0
39 2 1 1 2 0 0
40 2 2 2 1 1 0
41 2 1 0 1 0 0
42 1 1 1 1 0 1
43 2 1 2 1 1 2
44 1 0 1 1 0 2
45 0 1 0 1 0 0
46 2 2 1 2 2 2
47 2 1 1 1 0 1
48 2 1 2 2 1 2
49 2 2 1 1 0 2
50 2 2 2 2 0 2
Totals: 90 73 45 75 40 71
Percent: 90.0% 73.0% 45.0% 75.0% 40.0% 71.0%

Each result produced by the various systems is assigned a numeric grade based on a comparison with the optimal antiderivative as follows:

So a perfect score on the Charlwood Test Suite would be a 100. Note that simplification of results using functions and/or domain restrictions on variables was not done. In other words, the comparisons are based on the results a novice user would see.

Rubi home