The AI Alliance is proud to announce the Trust and Safety Evaluations Initiative (TSEI) at the Artificial Intelligence Action Summit in Paris.
The mission of the AI Alliance’s Trust and Safety Evaluations Initiative (TSEI) is to bring together the diverse research and development across the AI industry involving evaluations of AI models and applications that use them. Evaluations cover safety concerns, like avoiding hate speech, utility concerns like alignment of generated content to user goals, performance, and many other areas.
TSEI will define a comprehensive, widely-sourced taxonomy of evaluation interests incorporating the work of many organizations that are members of the Alliance, as well as outside of the Alliance. Based on the taxonomy, TSEI will catalog implementation of evaluations (which we call evaluators) available from many sources, as well as define a reference runtime stack for executing evaluators efficiently at scale. Finally, TSEI will build leaderboards that enable end users to browse, search, and filter for evaluations of interest, see how popular open-access models perform against them, and download different deployable configurations of the selected evaluators on top of the reference stack, for deployments in their own environments. Over time, the evaluations will be categorized not only by general purpose, but also by relevancy for particular domains and use cases. The ultimate goal is to make it very easy for developers of AI applications with little prior experience to easily find and deploy the evaluation tools they need for their use cases.
We welcome your participation. To find out more, visit the Trust and Safety Evaluations Initiative website. Visit The AI Alliance website for more information about this initiative and all the other activities of the Alliance, including how you can get involved.
