Explore Further

1. Not being blocked - https://serpapi.com/blog/how-to-reduce-chance-of-being-blocked-while-web/
2. Rotate User Agents - https://www.scrapehero.com/how-to-fake-and-rotate-user-agents-using-python-3/
3. https://www.scrapehero.com/ seems like a good reference to read more