All about K-Means

K-Means, is probably, one of the easiest to implement, it doesn’t even use any High school Math, like partials, slope etc, just Middle school Math is good enough. And it doesn’t even have a training phase, I doubt, if we can, even call it a Machine Learning algorithm K-Means is a clustering algorithm, as the word means, group the data into K number of groups. It’s like a Government wants to name a capital city that is equidistant from all other cities! in practice that may not be possible but what we try to do is minimize the sum of all distances from the capital city to the others. That is it, now add K number of capital cities and add closer cities to each K city, we have K number of capital cities! or K groups! Since we use mean distance to find the centroids, we call it K-means (a least squared Euclidean distance) The Logic:
  1. Take a random K centroids, assign each data point to the nearest K,
  2. Calculate the mean for each data point in K clusters,
  3. Use this new means as centroids repeat the  step 2 until the mean no longer has any significant change, typically iterate N time over.
As simple as that! Is it? Here is the Iteration 1 and Iteration 15, one can see the centroids move as we iterate Image courtesy Andrew Ng’s Machine learning that I took a while back!