In many cases, we may need to change or set the user-agent for a particular web application. When we browse a web application using Google chrome following is the user-agent it sets by default:
< navigator.userAgent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36"
When we browse a web application using OSX safari following is the user-agent it sets by default:
< navigator.userAgent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.1 Safari/605.1.15"
When Google bot or Facebook bot crawls a web application the user-agent is different. It is possible to customize the browser user-agent.
Why we need custom user-agent?
In our case, we have some premium content that Facebook wanted to get access to. We had to allow a particular Facebook user-agent to crawl our premium content. The Facebook user-agent was: Publication-Access-for-Facebook
Our premium content was under a paywall that was managed by varnish. We had to add the Facebook user-agent (Publication-Access-for-Facebook) on the varnish allowed list.
How do we test the custom user-agent?
Testing is a crucial part of this custom user-agent. The best way to make sure the custom user-agent is to add an extension that allows us to use custom user-agent. In our case, we have used the following Chrome extension.
Once we installed it on the Google Chrome, we were able to set the user-agent to our custom one: Publication-Access-for-Facebook
When the user-agent was set we found that the premium content is accessible from Google Chrome.
We can also check this using cURL by passing the custom user-agent. We can get more on this by using this article.