Module 1. Lesson 2. Crawling, indexing, and ranking

By | November 3, 2019

so let’s talk about how search engines actually work sure Google’s algorithm is extremely complex but in its simplest form Google is really just a pattern detection program when you search for a keyword phrase Google is going to provide you with a list of websites that matches the pattern that’s related to your search most people don’t realize that when they do a search on Google they’re not actually searching the live web they’re actually searching Google’s index of the web the stored copy of the sites that Google has crawled Google uses little programs called spiders to crawl the web the spiders are how Google actually discovers content basically a spider will start on a page and check out all the content on that page and then it follows the links on that page and looks at the content on those pages it follows the links on those pages to discover even more content and then the links on those pages lead to more pages and more content the spiders keep crawling at a mind-blowing ly massive scale until most of the Internet has been crawled as the spiders crawl all of these pages a copy is stored on Google’s servers this is Google’s index and it’s stored in a way that allows Google to quickly search through the billions of pages in the index when you search for a certain phrase Google checks the index to get a list of every page that’s related to your search phrase here’s where ranking comes into play this is the part that SEO is work to influence google uses an algorithm to look at that list of pages it just pulled from the index and rank the pages based on relevancy let’s say you did a search for Denver omelet recipe Google’s going to search the index and come up with a list of every website with a recipe for a Denver omelet then it’s going to use the algorithm to sort that list so that the most relevant sites will be at the top there are hundreds of ranking factors each with a different assigned weight or value the algorithm will look at a few hundred different factors that influence relevancy like the content on the page the number of other sites that link to that page and the overall quality of the website when we do SEO we’re hoping to influence those relevancy scores we know that if we optimize the right signals Google’s algorithm will decide that the page is more relevant that it’s a better answer to the question being asked and it will show the page higher than the other pages in the index

7 thoughts on “Module 1. Lesson 2. Crawling, indexing, and ranking

  1. ForeverZeroDragon Post author

    Nice and clear video.
    I think the books on the bookshelf are not yet indexed?

  2. tsungai Mubaiwa Post author

    Wow after a lot of searches I have understood the difference between crawling, index and algarythms. Thank you. All in less than 3minutes.

  3. santhose aghilsanthose Post author

    On first impression, i had no idea on what is crawler in search engine terms. But after seeing the video, i got good insights. Thanks for the information

  4. rafi malik Post author

    It's really wonderful to understand the crawling and indexing. Now clearly understand it. Thank You SEMrush.


Leave a Reply

Your email address will not be published. Required fields are marked *