A human designer probably won’t have the ability to capture all of these considerations in a reward operate on their first strive, and, even if they did manage to have an entire set of concerns in thoughts, it might be quite tough to translate these conceptual preferences into a reward operate the environment can immediately calculate. That is a very vital property for a benchmark where...