Announcing the Pitch Design Target Generator (Patent Pending)
I’m excited to announce the launch of the Pitch Design Target Generator service, where college teams, players, or facility owners will be able to send in pitch data from their pitch tracking device of choice, have quality of existing pitches evaluated by machine learning algorithms, and then receive suggestions for changes to current pitches and new pitches the pitcher may want to add to their pitch mix. The service will cost $129 per player per report. Those interested in this service should direct all inquiries to firstname.lastname@example.org.
Also, be sure to check out the updated web app which generates pitch design targets on Statcast data.
While all business questions should be directed to the e-mail address above, I will go into detail below on how the tool broadly works as well as the validity of the pitch scoring model.
The tool is straight forward. Pitch tracking data is input into a model which then uses machine learning to determine the pitch types the pitcher is currently throwing and assigns a score on a scale of 1-10 for each pitch, with 1 being a bad pitch and 10 being an elite pitch. The average and standard deviation of Pitch Scores for different pitch types in the web app linked above be found in the table below:
(Note: the averages and standard deviations in the table above are based on the version in the web app that uses Statcast data… the averages will be slightly different for reports generated for college teams since they are all generated by models’ prediction of pitch quality instead of performance from Statcast data. You will still be able to see how the pitch compares to other pitches of the same type since there is a scouting grade (20-80 scale) assigned to each pitch. This causes the pitch grades to be slightly off in the target arsenals in the Statcast demo version.)
The Pitch Score metric does not factor in command, consistency of the pitch, deceptiveness of delivery, etc. when assigned by the model, but it still ends up being a good measure of pitch quality and performance overall. More on that below.
Fastballs tend to have the lowest pitch scores and perform the worst while breaking pitches tend to score higher. The rate of both swinging and called strikes were among the variables included in the Pitch Score metric, so it’s not terribly surprising to see breaking balls score much better than fastballs. Despite whatever shortcomings which may exist in the Pitch Score metric, the section further below demonstrates it is still a great predictor of overall performance when weighted for pitch usage.
Once Pitch Scores are calculated for each pitch and pitch matches are found, the pitcher’s current and target pitch arsenal is plotted, and a table is created with current and target pitch characteristics. Text analysis of the pitcher’s current and target arsenal is also created and details the order pitchers may want to prioritize pitches in pitch design. The output looks like this:
The pitch grade metric is simply the pitch score adjusted for pitch type and placed on the traditional 20-80 scouting scale based on standard deviations below or above average (50 is average, 60 is a standard deviation above average, etc.). The pitch grade makes it easier to compare pitches of the same pitch type while the pitch score metric allows pitches of different types to be compared.
The auto-generated text analysis in the Statcast version may be a bit buggy, but this will be cleaned up in any report ordered by college teams.
Validity of Pitch Score Metric and Predictive Model
How good is the Pitch Score metric itself?
The original metric incorporates variables such as swinging and called strike percentages, batted ball metrics, and more. These variables were incorporated into a metric for each pitch thrown between 2015-2018 that met a minimum use threshold.
After that, I weighed the score for each pitch based on usage percentage for each pitcher to arrive at a score for the pitcher’s entire season and compared it to xFIP. Simply using the weighted Pitch Score metric to predict xFIP of the pitcher in the same season led to an R^2 value of .49 with a standard error of .6132 runs. Below is the output for the linear model in the R console and the corresponding graph:
Let’s look at it in one more way. Let’s say you wanted to predict a pitcher’s xFIP for the following season based on either their previous season’s xFIP or the previous season’s weighted pitch score. Which would you rather use?
The previous season’s xFIP is a better linear predictor, but not by much! The differences in R^2 values are small, and the standard error differs by less than .02 runs. So, the previous year’s weighted score is about as predictive as the previous year’s xFIP in predicting xFIP for the following year. The Pitch Score metric appears to be a good proxy for how good the pitch really is.
How good can a model predict the Pitch Score metric when variables such as swinging strike percentage, called strike percentage, etc. are not available?
The next step is to see how well the model can predict Pitch Score on a test set after training the model using variables which are available from most pitch tracking devices. The model was able to predict the Pitch Score on a scale of 1-10 with an R^2 value of .63 and a standard error of 1.48 points. See the screenshot below of the R console output for the model and the corresponding graph (Note: the pitch score and weighted pitch score have slightly different meanings in this model output, as the pitch_score below is the actual weighted score for the pitcher and the weighted_score is the predicted weighted pitch score):
(Note: the model tends to be conservative in assigning values close to 10 and 1 for the pitch score. Predicted scores are re-normalized on a scale of 1-10 and a new standard deviation is calculated for each pitch type to get accurate pitch grades, though as mentioned earlier, these will be slightly off for the target pitches in the Statcast version.)
A standard error of 1.458 points with a median close to zero means the predicted metric does a reasonably good job of predicting the quality of a pitch from data available from common pitch tracking devices. Those who end up incorporating reports from the Pitch Design Target Generator should be sure to note the Pitch Score isn’t directly factoring in command, consistency of pitch movement, sequencing, etc., so they’ll want to keep that in mind when engaging in Pitch Design and when determining optimal pitch usage rates.
Some things to consider
While the tool itself provides detailed descriptions of a pitcher’s current and target arsenal, it will ultimately be up to pitching coaches and the players to figure out how to create or alter the recommended pitches. The tool does not replace the pitching coach’s role in the pitch design process; however, it does provide a far more objective road map for how a pitcher can optimize their arsenal than currently available methods. This will allow coaches to be more confident in pitch design plans for each pitcher and allow more time to figure out how to materialize the recommended pitches.
The tool will work best on pitchers who already have reasonably high velocity (88 mph+ is a good place to start). Pitchers with lower velocities may not receive accurate Pitch Scores, and likely should be prioritizing velocity development over pitch design at that velocity anyway. Because of this, I will only charge and run reports on players who can hit at least 88 mph. Pitchers at lower velocities who need to develop a secondary pitch should aim to throw a high spin curveball or slider consistently.
Consistency of the pitch is also not factored into the predicted Pitch Score (in this current iteration, at least). If a pitcher’s average characteristics of a pitch grade highly but they struggle to throw it with a consistent movement profile, they will likely underperform their Pitch Score. This is another area where pitching coaches and players should weigh the tradeoffs of adding a new pitch. You may note some players with similar slider and curveball recommendations in the demo version, which is probably not advisable unless there is a sizeable velocity difference between the two pitches (this sometimes results from issues with pitch classification). I’ll be sure to manually remove these scenarios from any purchased reports.
While there are some rules in place to ensure different recommended pitches are distinct from each other, it’s not always clear how adding a new pitch will affect a pitcher’s feel for other pitches. When adding new pitches, coaches and players should be cognizant of changes to their existing pitches to ensure there aren’t any adverse effects.
The projected pitch scores are just that—projections. There are no guarantees on exactly how well the pitches will perform in-game. However, I’m confident the tool provides a far more objective road map for approaching the pitch design process than any other method currently being used. Ultimately, it will be up to coaches and players to make the most of the reports by figuring out how to bring the pitches to life and when to make judgment calls to deviate from the plan.
The tool provides insights a pitching coach or existing tools currently cannot as it uses a comprehensive process to score and recommend pitches quantitatively. I’m confident coaches and players alike will find it to be an extremely valuable resource when developing a plan for pitch design. Pitch tracking devices are inherently cool, but without a sensible way to understand and use the generated data to optimize a pitcher’s arsenal, they’re only expensive toys. This service provides badly needed context to data from these devices and will put players and coaches on the right path when diving into pitch design.
Interested in reports for your team? Contact email@example.com with the:
- Name of your team/facility/etc.
- Number of pitchers you’d like to receive reports for
- Pitch tracking device used to generate the data