K Nearest Neighbor Algorithm: Explained from Scratch.

  • KNN identifies the K number of neighbors.
  • It calculayes the nearest neighbors by calculating the distance. The distance between two points in space is calculated by any of the distances such as Euclidean, Minkowski, Manhattan(putting q=1,2 in Minkiwoski gives Manhattan & Euclidian respectively).
  • Now it does a probability test after calculating the number of neighbors.
  • In our case there are 2 blue and 1 orange. P(B) =2/3 >P(O)=1/3.
  • Hence the target is identified as Blue.
  • KNN don’t have any training process basically or because it directly does the calculations on test data by finding the Euclidean Distance for the nearest K points.
  • KNN can’t be used on big datasets because of the above point. If the data is very big then KNN will take a lot of time calculating the nearest neighbor of every point which is not at all practical.
  • When the clusters overlap KNN collapse because KNN works only on two dimensions same is with Naive Bayes also. SVM overcomes this problem.




Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Top 5 benefits of Continuous Internal Audits for Restaurant Food Safety

Random forest regressor used to predict combustion gas temperature using formaldehyde absorbance…

External Geospatial Data: Types and Use-cases

Exploring Data Analyst Roles

A Roadmap for Time-Series Analysis

Coursera IBM Capstone Project — The Battle of the Neighborhoods

Fully Utilizing Your Deep Learning GPUs

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Akshar Rastogi

Akshar Rastogi

More from Medium

SVM : Support Vector Machine — papalearn.com

Traffic Sign Classifier

K-Means Based Authentication

ReLU Activation Function