How can I use LOOCV in R with KNN? [on hold]

I am trying to use KNN with cancer data. At first, I only used separation data into train and test set, but I got unexpected results. So I want to use LOOCV to make sure.

I found only LOOCV with generalized linear models.

such as glm.fit = glm(mpg ~ horsepower, data=Auto)


glm.fit = glm(mpg ~ horsepower, data=Auto)


So how can I use LOOCV in R with KNN?

EDIT

My code

wdbc<- read.table("http://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/wdbc.data",sep=",",stringsAsFactors = FALSE)

wdbc<-wdbc[-1]

normalize <- function(x) {return ((x-min(x)) / (max(x) - min(x)))}

wdbc_n <- as.data.frame(lapply(wdbc[2:31], normalize))
wdbc_train<-wdbc_n[1:469,]
wdbc_test<-wdbc_n[470:569,]


I uploaded the data and I excluded the first column which is the class label. Then I separated the data into train and test set. However, I want to use LOOCV in the separation instead of my separation above.

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

separation

I want to use LOOCV in the seperation

caret

1 Answer
1

The knn.cv function from class package is based on the leave one out cross validation. The below implementation of this function gives you a LOOCV prediction of the full data (i.e. no separation into train and test).

knn.cv

class

full

library(class)

knn.cv(train = wdbc_n,
cl = as.factor(wdbc[,1]),
k = 4, prob = FALSE, # test for different values of k
use.all = TRUE)


Refer to knn.cv: R documentation

The general concept in knn is to find the right k value (i.e. number of nearest neighbor) to use for prediction. This is done using cross validation.

One better way would be to use the caret package to preform cv on a grid to get the optimal k value. Something like:

caret

library(caret)

train.control <- trainControl(method = "LOOCV")

fit <- train(V1~ .,
method = "knn",
tuneGrid = expand.grid(k = 1:20),
trControl = train.control,
metric = "Accuracy",
data = cbind(V1 = as.factor(wdbc[,1]), wdbc_n))


Output: fit

k-Nearest Neighbors

569 samples
30 predictor
2 classes: 'B', 'M'

No pre-processing
Resampling: Leave-One-Out Cross-Validation
Summary of sample sizes: 568, 568, 568, 568, 568, 568, ...
Resampling results across tuning parameters:

k Accuracy Kappa
1 0.9525483 0.8987965
2 0.9595782 0.9132927
3 0.9701230 0.9355404
4 0.9683656 0.9318146
........................
13 0.9736380 0.9429032
14 0.9718805 0.9391558
15 0.9753954 0.9467613
16 0.9683656 0.9314173
17 0.9736380 0.9429032
18 0.9630931 0.9197531
19 0.9648506 0.9236488
20 0.9630931 0.9197531

Accuracy was used to select the optimal model using the largest value.
The final value used for the model was k = 15.

qplot(fit$results$k,fit$results$Accuracy,geom = "line",
xlab = "k", ylab = "Accuracy")


wdbc_train< wdbc_n[1:469,], wdbc_test<-wdbc_n[470:569,]