From music recommendations to high-stakes medical treatment selection, complex decision-making tasks are increasingly automated as classification problems. Thus, there is a growing need for classifiers that accurately reflect complex decision-making goals. One often formalizes these learning goals via a performance metric, which, in turn, can be used to evaluate and compare classifiers. However, many useful metrics are combinatorial and non-decomposable; thus, standard algorithms and analyses do not readily apply. This talk will explore various statistical and algorithmic issues in classification with complex metrics by revisiting simple geometric properties of classifier statistics. To this end, I outline how this geometric view simplifies algorithms for consistent classification, elucidates the properties of fair classifiers, and enables efficient algorithms for metric selection using elicitation from expert feedback.