Cut off analysis is commonly used in Medicine through diagnostic tests that are used to discriminate between diseased and healthy populations using biomarkers analysis. In this case the cut-off value defines the positive and negative test results. Cut off analysis is also used in Finance especially in credit scoring where the risk of a loan applicant is evaluated. (For more information about financial cut-off analysis go to Hugo Lopes post https://medium.com/james-blogs/should-we-reject-reject-inference-an-empirical-study-4f1e5d86bcf4)
As opposed to finance and medicine I thought it could be an enjoyable challenge to develop a cut-off analysis on something a little unusual and close to my heart — Board games; specifically DnD. For those who are not familiar with D&D. D&D turns out to be the a fantasy table-top role-playing game that has been around for almost thirty years.
For a DnD analysis a dichotomous classification in positive or negative good or bad could be gotten from the Health Aka Hit Points of the character and the variable or score to analyze would be the AC Score (Armour Class Score) of one of the most epic games in history D&D also known as Dungeons and Dragons.
The base of this analysis will be built on a database already established by Burak Ogan (Repository oganm/dnddata). A criteria variable needs to be established initially which is created based on a rule-approach on one variable — In this case the hitpoints (HP — Health).
So for our analysis we are just going to define a character as good or bad based on the HP that the character has. If it has more HP than the median of the population we will define it as good otherwise it will be defined as bad. This has nothing to do with the alignment of the character read nature of the character law vs chaos. This is a way of having a dichotomous outcome so we can play with the AC score and stablish a hypothetical cut-off based on that.
Disclaimer: The results of this do not actually determine if the character is good or bad in the game. D&D was created in a way that there are not good or poor choices of characters. Every chracter is unique in their own way, there are different classes that you can choice from and basically the player is the one who can make the most out the character skills.
Say in D&D — Wrath of Ashardalon if you were to choose between VISTRA and HESKAN you will be deciding between AC and HP vs SPEED. If you go with the VISTRA you will have an Armour Class of 17 with some good 8 points of health HP but your SPEED the number of squares you are allowed to move will be lower than if you go with HESKAN 5 vs 6 even when the other stats from the chracter are better. This is without taking into consideration the Powers from you can choose from and the Special Ability each character has.
Firstly we are going to install the package that contains the database from github. The database we will be using is the one called dnd_chars_all.
The dnd_chars_all has 5805 characters. For doing a quick and pretty basic with emphasis in the basic part exploratory analysis the functions summary, boxplot and hist were used over the HP variable which represent the overall health of the character.
par(mfrow=c(1,2))boxplot(dnd$HP, main=”Boxplot of dnd$HP”, ylab=”dnd$HP”)hist(dnd$HP,breaks=100)
For segmenting the characters in good or bad a rule-base segmentation was used over the HP variable. If the chracter HP(Health) was over the median of this population it was classified as good otherwise it was classified as bad.
medianHP<-summary(dnd$HP)[]dnd$seg<-ifelse((dnd$HP<medianHP) == TRUE,1,0)head(dnd)
For the cut-off analysis over the AC variable the package(cutpointr) was used.
This package develop by makes really easy to do determine the optimal cut-off point.
For this analysis and for getting a hyphotetical point for AC two common metrics for ROC analysis were selected: youden and absolute sensitivity and specificity.
Youden is a really common metric that adds the sensitivity and specificity together and then deducts one out of this. While the absolute metric is the absolute difference between the specifity and sensitivity.
library(cutpointr)cutpointr(x = dnd$AC, class = dnd$seg)opt_cut_ydn <- cutpointr(dnd, AC, seg, method = maximize_metric, metric = youden)summary(opt_cut_ydn)
opt_cut_abs <- cutpointr(dnd, AC, seg, method = minimize_metric, metric = abs_d_sens_spec)summary(opt_cut_abs)
So basically according to our analysis playing with the Armour Class Score for the D&D characters contained in Burak Ogan database: The optimal cut-off point for selecting a character in D&D based on the HP is 16.
Following this if we want to guarantee having a “good” character selection based on the assumptions we made over the Health of the character -HP we should select a character with at least 16 in Defense AC.
Again,The results of this do not actually determine if the character is good or bad in the game. D&D was created in a way that there are not good or poor choices of characters. Every character is unique in their own way, there are different classes that you can choice from and basically the player is the one who can make the most out the character skills.
This was all made to have a dichotomous outcome so we could play with the AC score and establish a hypothetical cut-off based on that. In a real world exercise the criteria : positive/negative or good/bad would be based on performance. In the case of a medical analysis for example if we are testing the sensitivity of the skin over some chemicals we would look at cytokines that are a group of proteins secreted by specific cells of immune system (See ChungOp et al. paper over Optimizing the cutoff for the identification of skin sensitizers in ScienceDirect). In finance for establishing a credit cut-off score we would check the credit history of an individual(See Cut-Off Score in Investopedia or Hugo Lopes article in Medium).
So that is all from a board game enthusiast. I hope the analysis have been fun to read how it was to write. I think is always fun to find how to apply stats in daily life things specially if is something you like as board games and it is always refreshing to see something in a non-traditional approach.
The full analysis as well as the rmd document can be found here.