Indicators on slot You Should Know
The value with the identify attribute (robots) specifies the rule relates to all crawlers. the two the identify along with the written content characteristics are scenario-insensitive. to handle a specific crawler, switch the robots value of the name attribute While using the person agent token of your crawler that you're addressing. Google support