How to do the IP, user-agent whitelisting?

What is whitelisting

Whitelisting is a mechanism of allowing access to particular resources based on some criterias.

Example of whitelisting

Let say we have premium content that we want to allow access to a particular IP address. The Google crawler bot has a particular user-agent and we want to allow Google bot to crawl our premium content.

How many ways whitelisting is possible?

We can do the whitelisting in different ways.

  • IP whitelisting
  • user-agent whitelisting
  • Hostname whiltelisting
  • Custom HTTP header based whiltelisting