Claude Shannon in 1948 introduced his paper “A mathematical theory of communication” that had the novel ideal of information entropy. Entropy is a concept from physics. It is a measure of the extent to which particles within an object are active. If we consider the 3 states of water namely ice, liquid and vapor for example; we can see that the particle kinetic energy is highest in vapor and least in ice. This same concept is applied in mathematics ...