AdvertBench is a tool for ranking the capabilities of language models at generating image ads. This is a useful proxy for their ability to use Linux tools, create visual assets, and inspect their own work before finalising. Image models are not tested, since we're purely interested in the models' skill in using tool use and ability to read images.
Each ad is generated by an AI model running inside an E2B Linux sandbox through OpenRouter. Each model is given the same prompt and must use shell commands to create the files. If the model supports image input, it can use an image viewing tool to inspect the ads before finalising.
Since the quality of a generated ad is subjective, AdvertBench uses Elo rating system. Voters compare two ad sets generated for the same prompt, without seeing the model names or scores until after they vote.
To ask for a model to be added to the benchmark, email contact address.