The Parisi theory of the Sherrington-Kirkpatrick model completely describes the geometry of the Gibbs sample in a sense that it predicts the limiting joint distribution of all scalar products, or overlaps, between i.i.d. replicas. One of the main predictions is that asymptotically the Gibbs measure concentrates on an ultrametric subset of all spin configurations. Another part of the theory are the Ghirlanda-Guerra identities which in various formulations have been proved rigorously. It is well known that together these two properties completely determine the joint distribution of the overlaps and for this reason they were always considered complementary. We show that in the case when overlaps take finitely many values the Ghirlanda-Guerra identities actually imply ultrametricity.