Information Entropy and Learning

I recently came across the concept of information entropy, which is a measurement of surprise in learning some information...