BACK TESTING/WAGER TESTING and SUPERTUNING/SINGLETUNING
=======================================================
You can access the Backtest/Supertune features using the "Handicapping_Tools" Menu Bar item.
This HELP can be reached either when selecting BACKTEST or while SUPERTUNING. The operations are similar for both, and the SUPERTUNE/SINGLETUNE feature selected is identified by the information on the TOP line of the QuickDog Display. If there is no message (in RED) shown, then you are using BACKTEST.
The "Backtest" feature allows you to test to see how good your method was during a specific period of time in the past. The idea is that if your method worked in the past, then it is more likely to work in the future. The converse is also true, if it doesn't work in the past, there is no reason to think it will work in the future.
BACKTEST is also used with QuickDog's SUPERTUNE feature so that SUPERTUNE can automatically "tune" your handicapping method for you in order to produce the BEST backtest results. For more on SUPERTUNE, click on a "Weighting Factor" button located along the top of each column on the handicapping display.
To cancel BACKTEST, simply click the RIGHT MOUSE BUTTON. You will be returned to the handicapping method display.
Backtesting can be performed on all races within a given time period, or a breakdown of those races based on Performance, Distance, and Grade. When you select "BACKTEST", QuickDog looks at the current race you are viewing and sets its PERFORMANCE, GRADE, and DISTANCE settings. On the BACKTEST menu, you either select these specific settings or you tell QuickDog to use ALL races. Based on your selection, QuickDog will look for back races of the same type and at the same track. To do so, OBVIOUSLY QuickDog must have the results of those races selected. If not, then you will be alerted to that fact. Assuming the results have been downloaded, these race results are compared to the "picks" produced by your customized QuickDog method, and these statistics are listed so that the QuickDog selections can be compared to random chance. Stronger results indicate a stronger method. Again, this is obvious.
****NOTE: In order to use BACKTEST, you MUST download RESULTS CHARTS matching the programs you download. If you run backtest and see all ZEROS, this indicates you haven't downloaded the results charts.*****
You may also backtest wagers using QuickDog. For more help on this topic, click on one of the two wager buttons at the bottom of the BACKTEST display, and then select the HELP button.
BACKTEST uses all the ".prg" files in your \jaidogs\programs folder since the last download of racing data operation was done. If you delete these files, you will have fewer and fewer races to BACKTEST.
Also, BACKTEST will select the .prg files based on the most recent dates to the oldest dates and allow you to select these dates for checking a particular range of dates, i.e., from 06-01-2008 back to 01-01-2008, or from 01-01-2008 forward to 06-01-2008.
To keep your disk from becoming full of unwanted program files, you should trim the number of programs in this folder to keep things current. Simply put, after collecting a good number of programs, simply start deleting the oldest .prg file from the \jaidogs\programs folder as you add one to the \jaidogs\inpfiles folder. Also, you will want to delete the ".TAB" files in the \jaidogs\inpfiles folder which matches the .prg file you deleted.
REMEMBER - You MUST also download results files for BACKTEST to work at all!
What types of races are included in back testing?
=================================================
Depending on the method, races are included or excluded in testing based on the following:
All races that have dogs that have been scratched are thrown out. All 9 dog races are thrown out.
Why? Because this disturbs the true comparison between the way a method performs and random chance. If QuickDog included races of less than 8 dogs, then random chance would be better than 12.5% for a win, for example. So, ensuring 8 dogs run keeps the statistics honest.
METHOD (DATA DISPLAY METHODS A, B, and C)
=========================================
When backtesting, you will notice that the columns shown on the handicapping display are also shown on the BACKTEST window in a table. As it backtests, QuickDog computes scores and fills in this table with information as depicted in the following paragraphs.
You may display results of BACKTEST using either Method A, Method B, or Method C. Each is described below.
Method A -
This method corresponds to the type of report mentioned in Bill McBride's latest book on handicapping the dogs - "How to Beat the Dog Races". For each of the method columns, ie, SPEED, EARLY, etc., it will list the percentages that each of QuickDog's picks finished in the position NOTED in the Button to the RIGHT of the Method A button.
For example, if you set the value to 1 (finished 1st) then the Column 1 for SPEED will give you the percentage of the time QuickDog's first pick finished 1st. In column 2, the percent of the time that QuickDog's 2nd picked finished 1st will be shown. This is continued across the display. Obviously what you'd like to see is that the first 3 columns include most of the WINNERS. If this is true, then this measurement can be used as a predictor.
If you wish to see how many times the picks finished 8th, then set the value to 8. Now you would want to see very low percentages when viewing from left to right.
Based on the measures of strength shown, you might wish to disable some columns which don't seem to work that well in past races, and enable others to be used in your method. Then go ahead and SUPERTUNE those handicapping weighting factors.
Method B -
Method B requires another input of 1, 2, 3, 4, 5, 6, 7, or 8 in the Button to the right of the METHOD Button. If you select the number 1, QuickDog will display the percent of the time that QuickDog's number 1 pick finished 1st (Column 1), 2nd or better (Column 2), 3rd or better (Column 3), etc.
Statistics would indicate that random chance allows you to pick a dog to come in "1st or 2nd" at 25%, so the "place" column will indicate whether QuickDog outdoes this expected measure.
The expected values in each column assuming your method is no better than purely lucky at picking the races are:
12.5 25 37.5 50 62.5 75 87.5 and 100%
If you select 2, the column results will give the percent of the time QuickDog's 2nd selection finished 1st or better (Column 1), finished 2nd or better (Column 2), finished 3rd or better (Column 3), etc.
Selecting 3, will test QuickDog's 3rd pick, selecting 4 will test QuickDog's 4th pick, etc...
Of course all of QuickDog's picks will finish at least 8th, so column 8 will always be 100%.
Method C -
Each column measure for the particular method is listed and QuickDog's percent success in its selection are shown on each line. For example, the first column shows how well the method pick's a dog to WIN.
Column 1 = % of the time QuickDog 1st pick finishes 1st
Column 2 = % QuickDog 1st 2 picks finish 1-2 in any order Column 3 = % QuickDog 1st 3 picks finish 1-2-3 in any order Column 4 = % QuickDog 1st 4 picks finish 1-2-3-4 in any order Column 5 = % QuickDog 1st 5 picks finish 1-2-3-4-5 in any order
etc...to Column 8
Column 8 = % QuickDog 1st 8 picks finish 1-2-3-4-5-6-7-8 in any order Always 100% For Method C, ONLY the NORMAL output is displayed, and DELTA is not selectable if you are using Method C. See FORMAT later.
DEFAULT Settings
================
Based on the date range and currently selected race's track, performance, distance and grade, the database will be searched for races of similar type to produce the statistics. However, you may tell QuickDog to use all dates, all performances, all distances, and all grades in its Back Testing or selectively choose among these races using the buttons shown and described below.
DATE RANGE
==========
Click on the button to the right of the words "BACKTEST CURRENT SETTINGS". A menu will appear which allows you to enter the dates of the programs that you wish to consider for backtest.
While entering a date range to backtest, QuickDog will allow two options. It can either backtest from TODAY and proceed backwards through time, OR, QuickDog can begin at a particular date, and move FORWARD in time. To determine which direction to backtest in, use the FORWARD/BACKWARD button on the "PROGRAM DATE RANGE" display.
For example, you might select OCT 20 2001 for your beginning date, then select NOV 20 2001 as the most recent program you wish to examine. After you select these dates, press the RIGHT MOUSE BUTTON to return to the backtest window. Only programs within these date ranges will be used NO MATTER how many races you've asked for.
After making your selections, click the RIGHT MOUSE BUTTON to return to the backtest display. You should then notice that the dates you selected are shown on the BACKTEST display.
USE PERFORMANCE
===============
This is one of 4 buttons designed to filter the races considered for Back Testing. PERFORMANCE means "Matinee" or "Evening". By toggling this option, QuickDog will only backtest races of the same performance as the program you had displayed before selecting BACKTEST. Select again, and it will say "ALL", meaning ALL Performances will be used for BACKTEST.
For example, prior to selecting backtest you are looking at a Matinee Program. If you toggle USE PERFORMANCE you will see "M", then BACKTEST will only look at Matinee Programs.
USE DISTANCE
============
Toggle this button to use only those races that are at the same distance as the race just displayed before you selected BACKTEST. Or set it to use ALL distances. For example, prior to selecting BACKTEST you were looking at a 5-16 of a mile race. If you toggle "USE DISTANCE", then BACKTEST will only look at races that are at 5-16 of a mile. It will display "5-16".
USE GRADE
=========
Toggle this button to use only those races that are at the same grade as the race just displayed before you selected BACKTEST. Or set it to use all grades. For example, the race you were looking at before selecting BACKTEST was a GRADE C race, then if you toggle "USE GRADE", then BACKTEST will only look at GRADE C races. It will display a "C".
By setting combinations of these three buttons you can examine many different cross-sections of races. For example, ALL races, ALL 5-16s, or ALL 5-16s Grade C. This may point out better or worse percentages of win, for example.
NO. RACES
=========
This field is the maximum number of races to be used. Upon entry to the Back Test feature this value is set to 1000 (default). You may change this default value by selecting this button and entering the new number into the calculator and pressing the RIGHT MOUSE Button. The maximum number of races QuickDog will process is 9999.
While the Back Test feature is running, you will notice this field counts the number of races found matching your search criteria. It will also flash to indicate the Back Test feature is still running.
When it stops flashing and goes to yellow in color, the back test is complete. It then shows the number of races considered. The max number of races to consider HAS NOT CHANGED however and there is no need to reset this number each time before running the BackTest.
SUPERTUNE ALGORITHM TO USE
==========================
As described elsewhere in the QuickDog help, SUPERTUNE will automatically create weighting factors for all the Columns of a Method. Since NOT ALL of the combinations of settings can be tested, there are 3 techniques you can choose from which control the way SUPERTUNE operates. The basic SUPERTUNE algorithm is to tune each weighting factor in turn, and if the method is improved, backup and retune each preceeding factor again. This is continued until NO MORE improvement is made. But, the question is WHAT ORDER is this tuning done in. Well, typically from the 1st Column Weighting Value to the last as shown on the Handicapping Method. However, the user may choose 2 other orders that this might be done in.
In other words, there are 3 ways to ORDER to way in which SUPERTUNE will tune your weighting factors.
First, you can simply tune in the order that the variables appear on this display from the first to the last - "TOP TO BOTTOM".
Second, you might want to tune in order of the values (in column one of the BACKTEST DISPLAY) from the weighting factor that is PRIMARY (highest in value) to the one that is least useful (the lowest in value).
The third choice is to tune from the least useful, the the PRIMARY or most useful Column of the method based on the values in COLUMN 1 of this display.
FORMAT
======
The display defaults to displaying the absolute % hits on Win, Place, Show, etc., up to 8th place, based on your settings, type A or B. These values are obviously 0-100%.
However, sometimes it is easier to see DELTA values for a column. Therefore, we give you this option. When in DELTA format the value for a pure chance outcome (see above for those values) is subtracted from the value QuickDog computes.
For example, we all know that the chance of picking a dog to run 1st in an 8-dog race is 12.5%. If QuickDog measures its hit rate at 20%, then you will see a value of 7.5 in that column. If values are negative, this indicates a factor is not useful at all in determining good results, positive indicate the factor is useful (more useful than chance).
Using the Backtest feature doesn't provide numbers directly so that you can set these into your factors; however, it gives an indication of which factors are important. The more important the factor, the more weight should be given to that factor.
ANALYZE
=======
Once you have entered all your selections, you may RUN a BACKTEST by selecting the ANALYZE button. When you do, the names of the program files being analyzed will appear on the top line of the QuickDog window. You will notice that these programs are analyzed from the most recent, backwards to the oldest program until a sufficient number of races have been found which meet your selected criteria, or QuickDog has run out of race programs.
Finally, a message will appear on the top line to tell you that BACKTEST has finished this RUN. At this point, you may change any parameters and ANALYZE again, or simply click the RIGHT MOUSE BUTTON to exit BACKTEST.
PP WIN% (White lettering)
=========================
After running ANALYZE, you will notice that numbers appear in the row labeled "PP WIN%" above the columns numbered 1 through 8.
These numbers give the number of times a dog won the race from each of the post positions 1 through 8. The values are given in percent of the total number of races backtested.
For example, if 50 races backtested, and the dog in post position 1 won 16% of the time, and the dog in post position 2 won 20% of the time the first two numbers will be 16 and 20. You can use these numbers to quickly find a "hot post position" during a meet. You might even consider simply betting this post over and over again as a method of handicapping.
CHARTS
======
The chart feature will allow you to look at the trend of the race results during the period of BACKTEST.
When you select CHARTS, you can see the Win Percentage of each column of your method broken down by race date. Overall you can tell whether your method was improving or growing cold over a period of time.
You may also look at the trend of your BANKROLL, ROI, and Win Percentage of the Wagers you are testing. These results are "cumulative" and show how these figures grow or shrink over the time period charted.
Another chart feature allows you to look at up to 3 trends at once. Use the NEXT VALUE button to "lock in" a selected value and move on to the next one. Then select another value, and then hit NEXT VALUE again to move to the third value. At any time, you can select the PLOT ALL DATA to plot up to 3 selected values. Pressing PLOT ALL DATA again will display only the selected single value.
On the CHART display you may also create a "comma separated file" which can be imported into spreadsheets. This file will contain the data QuickHorse uses to do its charting.
------------------- END OF HELP FOR BACK TESTING ---------------
