Ads.txt: what’s the function of the file?

Lots of websites finance themselves through advertisements. Adverts allow website providers to generate revenue and make the media content on their pages available to visitors for free. A big market for programmatic advertising (fully automatic sale and purchase of online advertisement spots) has appeared as a result – and it sometimes handles pretty substantial amounts of money. So it’s no wonder that there are also plenty of black sheep romping about in this area, trying to get a piece of the marketing cake without any actual effort. With the initiative ads.txt, the Interactive Advertising Bureau (IAB) wanted to hamper attempts at ad fraud. Ad fraud refers to mock advertising efforts that either aren’t provided at all, or only to an insufficient extent.

The Interactive Advertising Bureau (IAB) is an international advocacy group for the online marketing industry. Their goal is to strengthen the industry, develop standards, and sustainably improve online advertising. Along with this, the association also endeavors to detect and prevent fraud within the industry. This concerns ad fraud in particular, which squanders huge sums of money every year. Both advertisers and honest publishers suffer from this type of fraud: While advertisers invest their money in scammers without receiving anything in return, website providers lose important revenue as cybercriminals pocket the money in their place.

Programmatic advertising functions automatically, for the most part: publishers offer advertising spots on their pages (e.g. on online magazines or blogs) and advertisers purchase or bid on these spots. The whole process usually takes place via a middleman – generally on a platform where sales or auctions take place. In this environment, cybercriminals frequently rely on the domain spoofing scam: advertisers believe that they’re buying an ad space from a renowned publisher (e.g. the online presence of an internationally recognised newspaper) – but the ads are later displayed on a completely different page.

The platform on which the ad fraud takes place doesn’t even have to be involved in the fraud. Often, the operator has no idea that scams are happening on their site. With ad fraud, cybercriminals feign that the website that they want to place the advertisement on is another, far more renowned one – in other words, they conceal the identity of their actual website. This is referred to as website cloaking. The website actually being offered usually has comparatively little traffic and often also has a bad reputation that could even cast the advertiser in a bad light. The ads.txt file is meant to prevent such scams.

IAB uses “ads” as an acronym for “authorised digital sellers” – although the term “ad” is more often used as an abbreviation for “advertising.” In this simple text file, the publisher should list out all providers that are permitted for the sale of their advertising space. The file gives publishers a greater say in trading their advertising space. At the same time, it also ensures more transparency, as it specifies exactly which providers are to be cooperated with.

Publishers deposit the ads file into the root directory of their web presence, the uppermost directory of the website. This is where robots.txt is located, for example, which provides the information for search engine crawlers. Ads.txt can also be processed by crawlers in the exact same way. The text file is publically accessible and can be read by machines as well as humans.

Since the file is located in the root directory and is required to be called ads.txt (example.com/ads.txt), it’s pretty quick and easy to find: For example, the ads.txt files of The New York Times, The Guardian, or The Washington Post can be called up with no further ado. This way, advertisers and platforms can check whether the sale is even permitted by the merchant in question. If a publisher is listed on an advertising marketplace and the corresponding ads file can be checked by all those involved, then it’s significantly more difficult for scams to run on platforms that spoof a false identity that’s not listed in the ads.txt.

With ads.txt, it’s now verifiable whether an offer is real or not. Some marketplaces, such as Google, have already implemented the standard on their own platforms. Ads.txt is automatically crawled if a publisher has made their inventory available. If the publisher hasn’t noted the marketplace in their ads.txt file, then no transaction can take place.

Since ads.txt is a simple text file, it can also be created with a simple text editor. There are three pieces of information that are necessary for authorised sellers – and a fourth that’s optional.

• Domain name of the seller: the domain under which the company that is permitted to sell the advertising inventory operates. The correct name can often be read directly from the provider or you can contact them to find out the correct domain name. For example, with Google offers, the correct name is always google.com.
• Account ID: if you’re registered with a provider as a publisher, then you have an account identification number. The same ID is also given for transactions.
• Relationship to the provider: you can enter two different values for this field. If “DIRECT” appears here, then the publisher undertakes management of their inventory on the platform themselves (under the ID that’s entered in the second field). The entry “RESELLER” signals that the publisher has charged a third party with the management.
• Certification ID (optional): if the advertising system is listed with a certification body, you can name the corresponding ID here. One such body is the Trustworthy Accountability Group (TAG). If an entry is submitted here, you would enter the TAGID.

All of these fields are entered one after another within one line, with the individual fields separated by a comma. If you want to include explanations in the file that aren’t meant to be considered by bots, you have to exclude them with hashes (#). Everything from the hash to the end of the line will be ignored. So if the character is put at the very beginning of the line, then the entire line will not be considered.

In addition to the seller entries and comments, the standard also allows two other pieces of information: “CONTACT” and “SUBDOMAIN.” Both values are entered into the file in the “Variable=Value” format, for example, “CONTACT=example.com/about_us.” While the first entry specifies a contact option, the second refers to an ads.txt file in a subdomain. Without the note on this second text file, bots would overlook it, because only the root directory of the main domain is usually crawled.

Whether you make an ads file yourself or prefer to have it created, how it should be made depends on how many platforms you work with. In principle, though, it doesn’t take much effort. If you’re a publisher and would like to protect your advertising inventory from fraud in the future, you can create the required text file in just a few steps. To do this, open a text editor (like Editor on Windows or TextEdit on Mac, for example) and enter the required parameters for the seller you’re working with. In our example, we’re working with Google for the advertising spaces that we offer on our fictitious blog.

Here, we confirm that we have two different Google accounts. Account IDs on Google always follow the same pattern: “pub-” (publisher) followed by a 16-digit number sequence. Some publishers have multiple accounts on platforms, if – like in our example – they’re logged into both AdSense and Ad Exchange. We can manage our own inventory in both accounts, and since Google has a unique TAGID, we also enter this.

To improve the clarity and provide interested parties with more information, we’ll specify exactly what the accounts are in the form of comments. We’ll also provide you with an e-mail address where you can contact us.

While the order of the individual parameters is already set, you can choose the order of the authorised sale platforms for yourself. For example, some publishers split their lists so that all “DIRECT” entries come first, and then all “RESELLER” entries. Within the individual parameters though, there shouldn’t be any spaces, tabs, or commas. If these are necessary, for example because part of the ID has a space, then you need to use URL encoding: This method, also known as percentage coding, uses the hexadecimal code of the character in the ASCII character set and places it directly after a percentage sign. For example, a space is represented by the character set “%20.”

When saving the file, make sure that you’ve stored it as a .txt file with the name “ads.” This is only way that ads.txt will be uploaded to the root directory of your server. This is possible with an FTP program, for example. The root directory is hierarchically the highest. Now you should be able to access the file under example.com/ads.txt.

Using an online generator can be worthwhile, especially if you work with lots of platforms. Some of them offer the service in exchange for your e-mail address, while others – such as the generator in Google’s DFP network – only add the entries to the text file for your own marketplace.

With an ads.txt validator you can check whether your authorisation file was correctly created and no errors crept in. For most validators, it’s usually sufficient to enter the URL. Some web services also offer an option to directly upload the file and so check it before publishing it on your own website. The ads.txt validator from AppNexus, a programmatic advertising platform provider, even allows users to make changes to the file directly in the tool, check them right away, and then download the new file. You then just have to transfer the altered ads.txt to your server.

But an ads.txt validator only checks if the syntax in the entries is correct. The online tools don’t check whether the information is true. This is important for advertisers, though: it’s not possible yet to use a validator to check if the publisher actually works with the named platforms.

Like so many standards, ads.txt didn’t catch on right from the beginning. One definite reason for its initially restrained use in the industry was uncertainty: How would the file influence individual businesses? It was only when the heavyweight Google started supporting the initiative and insisted on using the file from that point on that ads.txt gradually prevailed.

One point that has created uncertainty within the industry is that resale of inventory is somewhat complicated. Providers buy advertising space from a platform and sell it on their own. This isn’t always fueled by criminal intent, but ads.txt prevents such behavior. Publishers have no direct connection with these resellers and often know nothing about the processes. As a result, the provider doesn’t appear in the ads.txt file. This has caused some resellers to target publishers, which in turn has generated a lot of discussion within the industry and is seen by many as an attempt at fraud.

A point of criticism for many sellers is the manual creation of the file: it’s not guaranteed that there are no typos. An error can quickly creep into the domain name, and then the publisher’s inventory can no longer be traded on the platform. This can cause both the seller and the publisher to miss out on large sums of money. The use of an ads.txt validator, though, can significantly reduce the risk of typos, while checking whether the domain is concealing a programmatic advertising marketplace. And ultimately, typos can also be avoided through careful attention to the process.

Another weakness of ads.txt is the missing tagging of the type of agreed upon inventory: publishers can’t use the text file to signal whether the seller may sell display ads, video ads, or both. This means that platforms can continue to pretend that display ads are actually advertising space for video ads, and so receive additional commissions. Just because a seller appears in an ads.txt, it’s still not guaranteed that they’re legitimate. The responsibility for not entering into an agreement with scammers lies solely with the publishers.

You should also consider that ads.txt isn’t an all-purpose solution. Sure, you can protect yourself very well against domain spoofing with the file, but other forms of ad fraud remain unaffected. The admittedly laudable initiative still can’t put a stop to the fraudulent tricks of impression fraud and click fraud, which still cost advertisers a large portion of their advertising budget.

Despite its weaknesses, the initiative is still widely accepted in the industry and will continue to influence the advertising market in the future. Through the support of huge market players such as Google or well-known publishers, ad fraud scams can at least be put to a stop, because all other competitors have to follow suit with the implementation of ads.txt – unless they want to be at a disadvantage when it comes to dealing in online advertising spaces.