How Google makes improvements to its search algorithm

By | February 23, 2020


Rajan Patel: The Google search algorithm
is made up of several hundred signals that we try to put together to serve
the best results for each user. Amit Singhal: Just last year, we launched over 500 changes
to our algorithm. So, by some count, we change our algorithm almost
every day or almost twice over. Scott Huffman: We really analyze
each potential change very deeply to try to make sure that
it’s the right thing for users. Mark Paskin: The first step
in improving Google Search is coming up with an idea. Scott Huffman: There are almost
always a set of motivating searches and these searches are not performing
as well as we would like. Ranking engineers then
come up with a hypothesis about what signal, what data, could we integrate into our algorithm. Amit Singhal: We test
all these reasonable ideas through rigorous scientific testing. Mark Paskin: The first is with raters. These are external people
that have been trained to judge whether one ranking is more relevant
and higher quality than another. Rajan Patel: We showed these raters
side-by-side for queries that the engineer’s experiment
might be affecting. We also confirm these changes
with live experiments on real users. Mark Paskin: And we do this in
something that’s called a Sandbox. We send a very small fraction of
actual Google traffic to the Sandbox. We compute lots of different metrics. Scott Huffman: In 2010, we ran over
20,000 different experiments. All the data from the human evaluation
and the live experiment are then rolled out
by a search analyst. Sangeeta Das: For each project, it’s usually one analyst
assigned from the moment that we’re talking to the engineers
trying to learn about their change. And the impact is quite small,
as you see. Mark Paskin: We then have
a launch decision meeting where the leadership of the search
team then looks at that data and makes a decision. Amit Singhal: That surely
we should fix. Sangeeta Das: Ultimately, the goal
of the search eval analyst team is just to provide an informed,
data-driven decision and present an unbiased view. Amit Singhal: OK? So, not approved. The team will understand
what’s happening. If our scientific testing says
this is a good idea for Google users, we will launch it on Google. Mark Paskin: For many years now, Google has been offering
spelling suggestions for queries that contained typos or misspellings. So, sometimes you’ll type
a query and you might see, “Did you mean,”
and then an alternate query. If you type a misspelling
of your medicine and you don’t click on
the “Did you mean” you might be getting results
that contain that misspelling. And they tend not to be
high-quality results. So, we thought about
a different kind of interface that we call “full page replacement.” And instead of “Did you mean,” you’ll see at the top of your page, “Showing results for.” And in the case
that we made a mistake, there’s another link,
“Search instead for” and it has the query that you typed. We call that link the “escape hatch.” For every time a user had
to click that escape hatch because the spelling algorithm
made a mistake, we wanted to make sure that
there were 50 other times that they got the right
spelling suggestion and they didn’t have to click
the “Did you mean.” And they were also looking to see
in the live traffic data how often were users
clicking on that escape hatch to make sure that the user signal
that we get from live experiments was lining up with the signal that
we get from our regular evaluations. We brought it to Launch Committee and based on the rater evaluations
and the live experiments, it was pretty clear that engineers had
done what they were supposed to do. And so we launched it. Amit Singhal: When you align
Google’s interest with user’s interest as we have aligned, good things happen. Mark Paskin: We’ve put a huge investment
into understanding what works for users. Rajan Patel: Is this change
gonna help users not only in the United States
or in English, but all over the world? Scott Huffman: I think we get excited when we feel like we’ve hit on an idea
that really helps a lot of users. Amit Singhal: Users keep
coming back to Google even though they have
a choice of a search engine every time they open a browser.

Leave a Reply

Your email address will not be published. Required fields are marked *