Since Summit—the world’s fastest and smartest supercomputer—has come on-line there has been a massive push for large scale AI that can leverage Summit’s unique capabilities. One of the challenges for AI researchers who are relatively new to HPC is how to communicate their research success with the broader HPC community whose stock-in-trade is FLOPs (floating point operations per second). This particular metric is rarely meaningful to the AI community; hence a disconnect. In this talk we provide concrete examples which demonstrate the inability of FLOPs to capture meaningful attributes of AI and offer suggestions for better metrics for AI based off recent experience.