Benjamini and Schramm's work on incidence graphs of sphere packings suggests a notion of conformal map between metric spaces which is natural under coarse embeddings. We show that such maps cannot exist between nilpotent or hyperbolic groups unless certain numerical inequalities hold.