English Google Webmaster Central office-hours hangout

By | August 11, 2019

JOHN MUELLER: –new people. Great. All right. Welcome everyone to today’s
Google Webmaster Central Office Hours Hangout. My name is John Mueller. I’m a Webmaster Trends Analyst
at Google in Switzerland, and part of what I do is talk
with webmasters like you all and help answer your questions,
any things that might be open, and bring your feedback back
to the teams here as well. As always, if one of you wants
to get started with a question, feel free to jump on in. GARY LEE: Hi, John. I just wanted to ask you– I’ve
seen some interesting situation with the servers at the
moment, and our site has dropped for a major
keyword to page two. I mean, we were at about
fourth or fifth place. We’re now at about 15th place. So something’s kind of changed. But at the same time, we haven’t
done any sort of marketing, and advertising, and
that kind of stuff to really boost the site. And the thing is that we’re
scared about what to do, number one. And second of all, we don’t
know who or how to approach it. And I know you’ve talked
about this before. But even Google’s
preferred advertisers that you’ve got with
AdWords, agencies and things like that are doing
underhanded tactics. Not underhanded. They’re doing bad
link-building practices. So I want somebody
to do marketing, to help us do what we need
to do for our business. But the same time,
everybody’s still stuck in this old
school kind of pattern of creating crap to
try and please us. And I just don’t
know where to start. JOHN MUELLER: Well, I
guess on the one hand, at least you have
an understanding, and you kind of see what
these people are doing. So you can kind of evaluate
what people are offering you. So I think that’s
a good first step. It’s hard to say what
exactly a site should be doing in general with
regards to marketing. I mean, there are lots
of possibilities there. With regards to the
search changes recently, I think these are essentially
just normal changes that we have, as
they always happen. When algorithms get
updated, things change. When the general structure
of the web kind of changes, those are the things that
we kind of pick up on. So I don’t really have
any concrete feedback where I can say, hey, you
should be doing exactly this, or going to exactly this agency. I know there are lots of
really good agencies out there. There are definitely
also some out there that are doing crazy stuff. So if you kind of have
an eye for the things that you know you
shouldn’t be doing, then I think you’re on
the right track there. GARY LEE: Yeah. I mean, the problem
is we’re still seeing all of our competitors
doing kind of the wrong things, and still getting away with
it, temporarily possibly, until another update. But by us not doing anything,
we really are losing out. So I just don’t know what to
look for, to even ask for. And I wonder if
there is something that you guys at
Google can do to help people kind of recommend
what they should be asking from a marketing company
on a very, very high level, rather than you should ask for
this, and that, and the other. But just like, this
is what you should be looking for in a company. Because this is a major part. If companies like us can find
the right people and the right processes, then so everybody
else for follow suit, understanding what it is
that they actually need. And I think this is the biggest
hole in the industry right now. Nobody knows where to
go or what to ask for. We’re sort of guessing and
doing the wrong things. JOHN MUELLER: Yeah,
I mean, we have a page on Help Center
about how to pick an SEO. That covers part of that. But it essentially
focuses on the things that would be like red flags. That if an SEO
company offers you guaranteed number
one ranking, then probably, they’re
doing something crazy. So those are the kind of
things we generally point at. But I’d definitely take
your feedback on board that maybe having something
more in the sense of, this is what you should
avoid, but also, this is what you should
be looking for to help people find the
right partners for this. I think that might be an idea. GARY LEE: Because you
got things like market outreach, and you
want to do some news, and you finish it out. But then the problem is that
these companies distribute your news to these
dodgy, you know, all kinds of
different news places. And then you happen to have
a link, just a URL link, at the bottom of that news
thing, and that’s gone out, and now you’ve built
loads of links. Or somebody else
takes on an article, and you’ve got a link
in that, and now that’s been put on their
blog, and somebody can say that you’ve built that. There’s just too many questions. Now I feel like I’m questioning
ourselves far too much and too rigorously, that we
don’t give ourselves a chance to actually
do anything. It’s just, it seems,
an impossible situation we’re in right now. But if we don’t do something,
we’re not going to move, and we’re going to
struggle as a company. JOHN MUELLER: Yeah. Finding the right
balance is always tricky. GARY LEE: Yeah. OK. Yeah. Well, I’ll let you go. But yeah, a weird
thing in the search to see the stats kind of
drop from a few others. And the results don’t look
too great, to be honest, John. And if you look at
the top 10, it’s not something that Google
should be proud of. JOHN MUELLER: So
you’re specifically talking about the ones
you’re looking at? GARY LEE: That’s
in virtual office. JOHN MUELLER: Virtual offices? GARY LEE: Yeah, virtual office,
or virtual office, London. JOHN MUELLER: OK. GARY LEE: Yeah. So especially the top 20. If you really add in
what’s on page two, it needs to be run through
another analyzer, or a bunch of people to look
through that and say– there’s things that
where– I can’t remember one of the names. It’s a really,
really poor result. And it’s basically, they’ve
called their back-end system a virtual office. And there’s nothing to
do with it whatsoever. So, yeah. JOHN MUELLER: OK. That’s good feedback. Yeah. I’ll pass that on to the team. NEMANJA DIVJAK: OK, John. Can I ask a question
about video site maps? JOHN MUELLER: Sure. NEMANJA DIVJAK:
Do we have to have a link to the original file mp4? They say that you have to have
title, description, thumbnail, and that’s it. But they require a link to do
a Flash file, or original mp4. So do we have to have it
inside it, linked inside? JOHN MUELLER: I’d look
at the specification. I’m not exactly sure how
that’s defined there. But as it’s defined
in the Help Center, it should be the way
that we process it. NEMANJA DIVJAK: OK. JOHN MUELLER: I’m not sure
100% which of those fields are required for
video site maps. NEMANJA DIVJAK: I’m
going to share it with you in one second. Sorry if it’s– MALE SPEAKER: OK. I know I did something
like that some time ago. A video segment for some site. I know that you have to have
the Flash file, or the file that plays on the default. You have to have a
few alternatives, but they’re not
mandatory, I guess. JOHN MUELLER: Yeah. There are definitely some
optional fields there. But if it’s specified as
required in the Help Center, then you should
really define it. And if you don’t have
it in your site map, then I think that
should also be flagged as an error when you submit
it to Webmaster Tools. NEMANJA DIVJAK: Yeah,
but there are no errors. JOHN MUELLER: OK. Then that sounds like you’re
doing everything right. NEMANJA DIVJAK: OK. Thanks. JOHN MUELLER: All right. Quick question about
the Panda update. There hasn’t been one
since end of October. We’re waiting for
our site to recover. Google shouldn’t really
delay or pause algorithms once it starts them. We do run these
things regularly. Sometimes, it’s more regular. Sometimes it’s less regular. But we tend not to preannounce
these kind of changes. So this is something
where I imagine it’ll just be run again
at some point again. So these things,
we try to keep them running as regular as possible. Sometimes, that’s
not always possible. But it’s not the case that we
artificially stopped them just to bug webmasters about that. Currently, we have
a site that operates international websites. However, do you know any issues
in georedirecting traffic based on channel? In other words, if I’m only
georedirecting organic traffic, and the rest of the traffic not. I guess from a
search point of view, there are a few things
you need to watch out for. On the one hand, if
you’re georedirecting, you need to also redirect
Googlebot like that. So if you’re redirecting
all users from the US to a specific part
of your site, then Googlebot, when it’s
crawling from the US, would need to be
redirected there as well. On the other hand, if you
use something like hreflang, you can use the x-default
for the georedirecting page, and for the individual
landing pages for the individual
languages and countries, you can specify that in
the hreflang, as well. The important part there is
that the individual country and language landing
pages don’t redirect. So you have your redirecting
homepage, for example, and then you have the
individual language versions that, essentially, anyone
can access directly. So if someone goes
to your home page, they get redirected
to the right version. But they can also go
to the right version directly if they
want to do that. And if you set it up like
that, then essentially, that doesn’t really matter, which
channels you handle like that. So if you redirect
traffic from other sources in different ways,
that’s not something we would really worry about
from a search point of view, but which might confuse users. If they go to your
website through search and they land on
this specific page, but if they go to your
website in a different way, maybe with a different
referrer, and they land on a different
page, then that could be kind of
confusing to them. But essentially, there’s
a blog post we did, I think like
November, around then, about handling international
home pages, which tells you a bit more about how
to set up to this redirector. Webmaster Tools now sends
out mass notifications on mobile usability alerts. Our clients are receiving
the mobile sites as Google’s new online
mobile friendly test. What’s up? Is it because our sites use
JavaScript-based redirects? I think first of all, if you
have some kind of redirect between your site,
make sure you’re using the markup
and the techniques that we recommend
in our Help Center, in the developer site for
mobile-friendly sites. That includes things like
the rel=canonical back to the desktop page. If you have a mobile-friendly
page that we see the rel=alternate for
the mobile-friendly page, that if you’re
redirecting by JavaScript, that we can at least crawl
the JavaScript and see that redirect as well. Ideally, you’d be redirecting
on the server side, not on JavaScript. Because everything
you do in JavaScript adds additional latency. So especially if you’re
on a mobile device, and you’re redirecting users
by JavaScript, then at first, they have to load your HTML. Then they have to load
the JavaScript file. Then they have to process
the JavaScript file. Then they see the redirect. And all of this adds up. So you’re easily
adding a couple seconds extra latency for
anyone who’s using a mobile phone to
access your site. So that’s something
where I’d recommend using server-based
redirect if possible. But essentially, if you set this
up properly for your website, then we should be
able to recognize that your site has the
mobile site attached to it, and that if the
mobile-friendly site that you have there passes
the mobile-friendly test, then that should be OK. What might also be happening
there is that you have some pages that pass this
test with the redirect, and other pages that don’t. So we’ve seen that,
for example, I think with our help form,
that was a case, that we had a bunch of the template
pages actually redirect you to the mobile-friendly
version, but there was one set of
template pages that didn’t have a mobile-friendly
page that we somehow forgot. So like 70% of the site had
proper mobile-friendly pages, but there was a large chunk
of the site that still didn’t have mobile-friendly
alternatives. So what might be
happening in this case is you have most of it set
up properly, but some of it still isn’t doing it right. And you can see that
in Webmaster Tools with the Mobile
Usability Report, where you’ll see a
graph of the issues that we found across your site. And you can drill down to look
at the individual pages there. So that’s another thing
I’d recommend doing. So first, make sure that you
have it set up technically correct with the redirect,
so that we can pick that up properly. And then make sure
that you’re not missing a subset of your
pages that are still stuck in this
unmobile-friendly state. I see a lot of the same
faces in these Hangouts. Can you limit them and
throw them all out? It looks like we have some
different faces today, so I will refrain from
throwing everyone out. But as always, I try to give
people a chance to sign up and say they’d like to join
these Hangouts as well. And if people let me know
about that ahead of time, I tend to add them a few
minutes before we start. So if you’ve really been
wanting to get in here, and you’ve never had a
chance to get through, then feel free to sign up there. ROBB YOUNG: John,
have you considered changing the times of them? Have you found a
difference in the people who turn up when
you change the time? Or are you limited on your
schedule because of that? JOHN MUELLER: I try to
take something that’s like, early morning
European time that works for Asia as well. And something a bit
later European time. But I know some people here
join in even at crazy hours during their time. Right, Gary? GARY LEE: Twenty minutes past
2:00 here in the morning. JOHN MUELLER: Yeah, so I think
if you really want to get in– JOSHUA BERG: I’ll
be here any time. JOHN MUELLER:
–there is a chance. But I totally
understand the point that sometimes, we talk
about the same topics over and over again. And it would be nice to
get some new faces here. But again, if you really
want to join these, just let me know ahead of time,
and I’ll add you a bit earlier. And everyone is refreshing
and jumping in too. MALE SPEAKER: Can I ask
a mobile-related question since we were on
the mobile topic? JOHN MUELLER: All right. Go for it. MALE SPEAKER: So I
have a few clients that I’ve been noticing a
few trends on the– they have a separated sub-domain
for mobile sites, that classic m dot. And unfortunately,
I’ve been trying to move them to either
dynamic serving or responsive. But in some cases, there is
just a bit of, I don’t know, confusion how they look at the
performance of these domains in Webmaster Tools,
like the desktop version and the mobile version. Because if you add both of them
to Webmaster Tools for both of the versions,
mobile and desktop, you have the option
in search queries to filter mobile and desktop. So once you add the canonicals
to the mobile version and everything is
set up properly with alternates pointing
to the desktop version, and the canonicals to the–
I mean, the other way around. The canonicals put into
the desktop version, and the alternates pointing
to the mobile version– what’s the expected behavior there? Because it’s a bit confusing. I’ve seen some sites
that start dropping in search queries for the
mobile version for the web, but then they show an increase
for the mobile on the desktop. And for other
sites, the behavior is not that, and although
everything is set up correctly. What’s the expected behavior
there if everything is correct? JOHN MUELLER: I
don’t know for sure. I haven’t actually looked
into that in detail. But I can double check. I know to some extent,
it’s a bit confusing, because some of
the impressions are clicks we track on
the desktop page because we think
it’s the canonical. But at the same time, if the
mobile page gets the traffic, then when you track the
clicks, maybe there. I would have to take
a look with the team to see what should
be happening there. MALE SPEAKER: Yeah. OK, because I looked– JOHN MUELLER: Because
it should be consistent. MALE SPEAKER: I looked in
the Web Help Center trying to look for some clarification,
and I didn’t find any. So that’s why I came
and bugged you here. Because I mean, when I’m
on the mobile version, and I filter on mobile,
what am I looking at? If I’m on the m dot
of my site, and I go to filter on mobile traffic,
what does that represent? And what does web
represent then? JOHN MUELLER: Yeah. I don’t know. I’d have to take a look with
the team at the details there. I know we looked into that
a bit for the redesign that we’re working on. So I hope that’s a
bit clearer there. But I’d have to
take a look exactly what the current status. MALE SPEAKER: OK. JOHN MUELLER: And I guess
if you guys haven’t seen it, I put out a call for a sign
up for the alpha versions of an update we’re doing for
the search query feature. If you haven’t signed up
and want to take a look, the old feature
will still be there if you sign up, by all
means, go ahead and do that. I think we’ll probably leave
it open for another week or so, and maybe get in touch with
you guys next week or the week after that to preview
the new version. MALE SPEAKER: I signed up
for all the tests, so– JOHN MUELLER: Fantastic. It’s still a very
early version, so I wouldn’t assume that it’s
going to look exactly like this when it
actually gets live. But we’d really love
to see your feedback on things like the usability. Are you finding the
right information? Are you able to
figure stuff out? Does it encourage you to click
around and try things out to find new ways of
looking at your data? I wouldn’t look at
it in the sense of, I want to have exactly the same
view as I had in my old setup. So kind of take a step back
and see what you can see there, and we’ll be asking
for feedback as well. So that’ll be interesting. MALE SPEAKER: OK. I might have to leave
in a few minutes. So anyway, thanks for
your time, and I’ll be watching a few more minutes,
and then I’ll be going. JOHN MUELLER: Great. Thanks. MALE SPEAKER: Thanks, John. JOHN MUELLER:
Thanks [INAUDIBLE]. ROBB YOUNG: And where are
those beta versions again, where there’s– JOHN MUELLER: I put the sign up
on my Google+ account somewhere in the– ROBB YOUNG: In
the last– yeah, I remember you mentioning
it on the last call. JOHN MUELLER: With
the big robot. It’s from the Google
Webmaster’s post that we shared. ROBB YOUNG: All right. Thanks. JOHN MUELLER: OK. What markup are exactly needed
to have a description and link shared on Google+? Some pages shared on Google+
have description below the image and title,
and some don’t. And how do we reload
the cache for this when we do changes in markup? We have, I think, on the
Google+ Developer side, there’s markup for a Google+
snippet that you can put on your pages that
we pick up on. So we pick up some of the things
like the Open Graph markup as well. But there’s also a specific
structured data markup you can do for the
snippet, where you can also specify the image
that you want to use, if you have a bigger
image on the page. So I’d take a look at that. I can pull out the URL of that
after the Hangout if you want. But it’s with the
Google+ Developers side. From January, some site owners
are faced with weird problems, including me. When an article is
published, the original site doesn’t come up when
I search for a title. But after seven to eight days,
everything becomes normal. What and why is this a problem? Is there any solution? This is something I
see from time to time. And usually, what’s
happening there is that our algorithms
are picking up on lower quality signals of the
site that’s actually publishing this article, where we kind
of assume that the site that’s publishing the article
is iffy, and we’re more cautious around that. So we don’t pick up everything
as quickly as possible there. So that’s something
where you might want to take a look at and
see if you can really improve the overall quality
of your website, so that you can be sure that
Google’s algorithms really are able to trust
your site completely, and able to take
everything that you publish as quickly as possible. So for example, if you
have it in RSS feed and we can pick that
up, then chances are we could pick that
up within a couple of minutes of your
publishing it. But if we don’t know that we
can really trust your website, then that’s something
where maybe we’ll be a little bit more cautious
and kind of hold back a bit. So if you’re seeing
these kind of delays, then that’s something
where I’d really recommend taking a step back and
thinking what you can do about your website overall to really
make sure that it’s sending all of the right signals, and not
sending lower quality signals, looking kind of
sneaky in some sense. Sometimes we see
this, for example, with sites that rewrite
or scrape articles from other sites. I don’t know if that’s the
case with your website. I don’t want to blame you
for that specifically. But it’s something
definitely worth looking at. It’s usually less
of a technical issue when we see that happening. Why does Webmaster Tools
think that the robots.txt file is a web page? We have html errors for
the robots.txt file, and now it’s our only
mobile usability error. I thought this
was kind of funny. I had definitely passed it
on to the team to look at. But essentially, things like
the robots.txt files, site maps files, those kind
of things, are also normal files on
your server that we can show in search sometimes. So depending on how we ran
across the robots.txt file, it might be that we’re treating
this as a normal web page, and you can maybe find it in
search if you specifically search for it. It’s not going to be
the case that it’ll rank for any of your
normal keywords, because we have a lot of
more relevant information for the other pages
on your website. But since we have it
in our search index, we can show it in some
of these tools as well. I think specifically for
the mobile-friendly test, that’s something we should
probably exclude, maybe together with other
text files on a site. Because obviously,
if it’s a text file, you can’t add
mobile-friendly markup to it. But this isn’t something
I’d worry about. In the worst case,
what will happen is if someone’s searching for
your robots.txt file in Search and sees it in
Search, then it won’t have the mobile-friendly
label on it. But that’s just your
robots.txt file. That’s not the rest
of your website. Another thing you
can do if you want to have it dropped
from search is to use the X-Robots-Tag no index
header for these kind of files. So that might be something. If you really want to
clean things up completely, you can do that. In most cases, it’s not that you
need to do anything about that. But if you want to take
the extra step, go ahead. There are first reports
saying that mobile-friendly is a ranking factor now. Can you confirm this? And if yes, in which countries? And what’s the status here? So we’ve had various
mobile issues already be ranking factors. So we did a blog post,
I think, 2013 even, about various types of mobile
issues that we’ve run across, which we use for
ranking as well. Back then, those were
mostly technical issues, like wrong redirects. Or if it’s a Flash-based site,
for example, that’s something we could have used there. And with the new
mobile-friendly tag, we’ve said that we’re
definitely experimenting with ranking changes
there as well. So I would certainly expect
that things move more in this direction. And the thing to keep in
mind in this specific case is that since it’s about
mobile-friendly sites, it’ll be something that
smartphone users will see. People who are
using mobile phones. So it’s not the case
that if your site isn’t mobile-friendly, that
we’ll remove it from Search, or that we’ll remove
it from Desktop Search, or demote it in Desktop Search. It’ll be just
specifically for people searching on a smartphone, that
if we have other results that are similarly relevant, but
they are mobile-friendly, then we’ll probably show those
first in the search results. And I think that makes a
lot of sense for the users. And to some extent, this
email that we sent out about these mobile-friendly
issues that touches upon this as well is meant to
let the webmaster know that, hey, we’re going to
be taking this into account. If this is something
you want to work on, and mobile is definitely
a giant traffic source, then that’s something you
should focus on early, so that you don’t have to
worry about this ranking change when it comes later
on down the line. And these kind of changes would
be relevant for all locations. Essentially, anyone who’s
using a mobile phone sees the same issues. If the site doesn’t work
well on a mobile phone, then it doesn’t matter
which location they’re in. It’s essentially a
global kind of thing. Let’s see. There’s one about a blog here. Probably, you
already know there’s too much competition
between blogs. So I’m going to give
you a fresh example to understand my question. Let’s say a blog has 30
articles about Nexus 6. Another blog has five articles. And somehow, this
question got cut off. Maybe it’s somewhere
down the line. OK. Let’s see if I find
more of that later on. Why mobile site maps? I find mobile site
URLs as web pages, but not as mobile pages index. The important part
with mobile site maps is to keep in mind that
they’re specifically for feature phones. So for feature phone pages, not
for mobile smartphone pages. So if you’re using mobile site
maps for smartphone pages, then you’re essentially
using them wrong. I only use this for
feature phone pages, if you have specifically like
WAP/WML, that kind of content on your pages. Our site struggles with Penguin. After a cleanup, we earn
many new natural links, but our site is
still suppressed. Is there a time frame after
a site is considered normal and links an account
like a site that was never affected by Penguin? Essentially, we have
to reprocess the links to your site, which
can take a while. And then the Penguin
algorithm has to run again, which
is also something that can take a while
to be updated again. So this is something
where you’re not likely to see a big jump just
after cleaning these things up. It’s more gradual of a
thing, and this kind of ramps up again as you
clean up your site as you show us that
things are really well. And it’ll probably
take a bigger jump when the next Penguin update
happens, which at the moment, we aren’t preannouncing yet. Is that an SEO
factor, to rank higher the specific keyword Nexus 6? I didn’t quite
understand this question. I think this is a part of
the other one from before. If you’re watching along, maybe
if you can phrase the question in a brief way so that I can
see everything in one slot, that would be kind of helpful. When queries are not
revealed through analytics, are the unrevealed
queries the same keywords as the 10% approximately
revealed queries? Or are they unrelated
queries, sets of completely
different keywords? They don’t necessarily
have to be related. It’s not the case that
we take a 10% sample and just show that in analytics. For some types of setups
when people search, I believe we still kind
of pass the referrer on. And that’s probably what
you’re seeing in analytics. To get a bigger picture
view of what actually people are searching for,
I’d really recommend using Webmaster Tools with
a search query feature. Or if you want to try
out the new feature, make sure that you’re
signed up there, so that you can look
at that as well. But it’s not the
case that this 10% is like a representative sample. It’s essentially
just those queries where we happen to still
pass a referrer on. And that might be a specific
subset of your users, that maybe you’re using older
browsers, or special setups, or within specific
experiments in search. It’s not the case that this
is a representative sample of all of your queries. We’re an e-commerce
site from Nepal and intend to sell
homemade products in countries like
the US and Canada. Do we need a special sub-domain
or URL, like /CA, /AU, to rank in a specific country,
or is having the same website for each English country
essentially also fine? Essentially, you can
do this either way. So if you want to
use geotargeting, if you have something specific
for individual countries, if you have specific
information that’s kind of unique for those
specific countries, then by all means, you
can use geotargeting, which you can use if you have
a generic top level domain, like a dotcom website you
can use on a subdomain. Like us.yourwebsite.com,
or a subdirectory, yourwebsite.com/us, for example. You can set the geotargeting
up in Webmaster Tools for that. If you don’t have unique content
for each of these countries that you’re targeting,
you can also just use one global
website and say, this is our global website. These are the products
that we’re selling. And we deliver these to
these countries, for example. And that’s
essentially up to you. One advantage of
using a global website is that you’re kind
of concentrating your efforts on
that global website. You have less content to
maintain on your website. It’s a little bit easier
to keep up with everything. And it helps us to kind of
concentrate all of the signals for your website
for your business on that main global website. Whereas if you really
have unique content for individual
countries, then splitting it up might make sense. But it’s something where
you have to kind of keep in mind that having individual
countries on your website also means that you have more
content to maintain, more things that could
break at some point, that could go wrong on your
website for technical reasons. So it’s kind of a balance there. If you really have
something great that you want to
offer individually, then by all means, do that. If you prefer to get started
with one global website, then that’s also fine. And moving between
these variations is certainly possible. So if you start off with
a specific site for the US and Canada, and later
decide, I’ll just fold it into one global website
because that’s easier for me, then that’s something
you can also do. How does better testing a new
website on the same domain not optimized for SEO, and not
meant to be visible by Google on Safari only can affect SEO? Does having Google Analytics
tag on a new website give access to Googlebot
user agents by default? No. Google Analytics tags
don’t by default send that off to Googlebot. I think there are some things
that we might pick up on. For example, if you
have an RSS feed, and that happens
to be published, then that’s something
we might pick up on and use for Google search. But in general, if
you’re blocking access to those kind of preview
developer site, if you’re blocking it based
on the user agent, or based on the IP
address, or with a login, then that’s something
where Googlebot wouldn’t be able to kind of
get around that. And essentially, this
wouldn’t affect us at all. If we can’t crawl
those pages, then we can’t really pick those up
and use them for search. The only effect that I
think might be possible is if people are
linking to those pages already, if they’re recommending
them to their friends, then we kind of get
stuck, because we see these links
pointing to these pages. We can’t crawl them. So we don’t really know
what to do with those links. This is especially
more problematic if you use something like the
robots.txt to block access. Because in that
case, we won’t even try to crawl those pages,
because a robots.txt file would say, don’t crawl any of these
pages on my developer site. But at the same time, if we
find links to those pages, then we’ll say, well, we could
index these URLs individually based on the links
that we have there. So if you have a developer
site, and you’re blocking it with a robots.txt
file, then it’s possible that we’ll
index those URLs without having to crawl them. On the other hand, if you
use other types of blocking Googlebot, like based
on the IP address, based on the login,
those kind of things, then we would try to
crawl those pages. We’d see that we
can’t crawl them, and we kind drop them
from search immediately. But in that case, if
you’re blocking Googlebot with an IP address or
with an authentication, then make sure you’re not also
serving a robots.txt file, because then we’re stuck with
the robots.txt file again. So the best practice
is therefore kind of like a developer’s
site, or a testing site that you’re running is really
blocking by IP address, or with the server
side authentication, so that we can try to crawl. We’ll see that we can’t
get to the content, and we won’t index it at all. A question about forums. Is it OK to display
posts that have only 500 plus words in Google? Sure. You could do that
if you want to. Will that help eliminate
low quality topics? Can you give some general
tips for forums and SEO besides keeping the
forums spam free. So in general, I think you
have to keep in mind that when we look at your website,
we look at it overall. And if the overall quality of
the content that you publish there is lower
quality, then that’s something we might take into
account in our algorithms. And it doesn’t really matter
where that lower quality content is coming
from, if that’s something that you’re publishing
as the owner of the website, or if that’s something that
you’re publishing based on things that people have
submitted to your site, like on a forum,
or blog comments, like those kind of things. So essentially,
this is something where you’re responsible
for your website, even if its content primarily
comes from other people. So if you want to make sure
that your website overall is a really high
quality, then I’d definitely try to
take a look at what you can do to make sure that
the forum posts, for example, if you have a forum, are
generally of high quality, or that at least those that are
indexable are of high quality. So if you know that there
may be new forum users, and they’re not complying
with your rules, or they’re kind of dropping
spammy stuff on your site, and you can recognize
this somehow, be it by the number of words
that they submit, or be it because
they’re just new users and have been in your forum for,
I don’t know, less than a week, or less than a month, or
whatever frame you choose, then doing something like
allowing those pages to be crawled, but having a
no index on them really helps us recognize
that this is something you don’t want to have indexed. And we’ll therefore not
take it into account when we look at the overall
quality of your site. So if you can find
some factors like that, it might be the
length of the topic. It might be the
number of replies. It might be something like
the age of the poster. It might be a combination
of all of these things. And using that as a proxy
to automatically determine if this is probably a
high quality post or not, that’s something you could use
to trigger a no index or not. But this is something where
you know your website best. You know your forum best. You know your users best. And what works for
some forums might not work for other forums. JOSHUA BERG: Hi, John. I have a site map
question I’d like to throw in regarding an
anomaly in Webmaster Tools. JOHN MUELLER: OK. JOSHUA BERG: So I was
reviewing a site for a client, and they have a
site map, and we’re getting the warnings this
is blocked by your robots. Some of the URLs are
blocked by robots. So they have a main site map. That’s the main XML site map. And then the way
they had it set up is there’s three site maps
under that site map, or in that. Like the first one
has 50,000 URLs. So it says 50,000 warnings. 50,000 submitted, and
then 49,500 indexed. The funny thing is
none of the URL– and we’ve checked–
were blocked by robots. So something else
is going on there. And the second one also
has 47,000 warnings. So that’s odd. And then it also says
processed, originally gave an earlier process time. So it wasn’t clear that that
was the most recent process time out of 2014. But anyway, now it has a
more recent process time, and it’s still
showing those errors. JOHN MUELLER: So
what we sometimes see is that a site has some kind
of fluctuating robots.txt file in the sense that either there
is something on the server that sometimes sends
us the wrong one, or that maybe there’s a content
delivery network involved, where some of the nodes have
the right robots.txt file, and other nodes
have the wrong one. Another one we’ve sometimes
seen is that may be like, a developer site robots.txt
file is accidentally uploaded. These kind of things. You should be able
to see a bit of that in the robots.txt testing
tool in Webmaster Tools, where you can also see the status of
the previous robots.txt files. So I’d kind of
double check there, that there’s nothing kind of
fluctuating happening there. If this is just like
a one-time thing where accidentally, the wrong
robots.txt file got uploaded, then obviously, that’s fine. If this is something that
regularly flip-flops back and forth, that’s something
where you probably want to figure out what
exactly is happening there, so you can fix that problem. JOSHUA BERG: And all
URLs being indexed fine. JOHN MUELLER: Yeah. I mean, that’s
theoretically possible. So if the robots.txt
is temporarily blocking everything, then it
won’t remove the existing URLs right away. It’s something that will take
effect the next time we crawl. And if this, for
example, comes up one day out of the
whole year, then we’ll still show that as a
warning, as a problem, but it’s not that
it would negatively affect the indexing
of your website, because the next
day and all the days before, we’ll be able
to crawl normally. So it’s essentially
something that usually isn’t a critical issue. But if it’s like active
or not active from time to time, that’s
something you probably want to chase down and make
sure that whatever is causing that to happen is
kind of resolved. And sometimes, we’ve seen that
this same issue can affect web pages as well, where
like sometimes, you’ll see the normal homepage. And then, every 1,000th
time or whatever, the user will see a 404 page,
or a 403 page that says, you have no access to this page. And these are the
type of things where it’s really hard to
recognize manually, because you reload the page a
few times, and it all works. You don’t realize that once
a day, it’s a specific time. Maybe when you’re
doing a backup, the server sends
back a code that says, hey, nobody’s allowed
to access this server, or sends back the wrong files. Those kind of things. But with the– JOSHUA BERG: I found something
that when I click on it and go to it, go to the
errors, that it still has detected June 2013. And when we do
the test site map, it shows that there
aren’t any errors. So will the errors just
always show there like that? Or does that mean
there are still the same errors from
more than a year? JOHN MUELLER: That
should go away, then. So I mean on the one hand,
that means that your robots.txt file is probably OK now. So if that was in June 2013,
then that’s quite a while back. But that’s something
where I think we should be able to
handle that better. So if you can send
me the sample URL, then I can definitely pass
it on to the site maps team, so that we can see why we’re
still showing these errors. Because I think,
like you mentioned, that wouldn’t make much sense,
to show the errors from 2013. JOSHUA BERG: And so by detected,
that doesn’t necessarily just mean the first
time it was detected. Because I don’t have another
date there but June 4, 2013. So I mean, that
may be long gone. But it’s still just
showing the errors. JOHN MUELLER: I don’t know. That seems confusing. That seems like something
we could be doing better in Webmaster Tools. JOSHUA BERG: I’ll
send you the link. JOHN MUELLER: Yeah. That would be great. What would your advice
be as far as procedure and what to look for
on the server log file analysis, mainly regarded
to Googlebot activities, Googlebot always
downloading the entire file? Sometimes the size is smaller. Googlebot doesn’t always
download the entire file. So sometimes, it’ll send
a conditional header with the request, saying
if this file was modified after this date when
we last crawled it, then send me the file. Otherwise, it’ll just get
a response code saying, OK, this file hasn’t been modified. So for those kind
of requests, it would be completely normal to
see the file size fluctuate between the full size
and a very small size, where basically we’re just
getting the header back that’s saying, nothing
has changed here. You don’t have to fetch
the whole file again. With regards to the log file,
what you should be tracking, I think it’s kind of up to you. It depends on how far you
want to dive into this. Obviously, depending
on the website, analyzing a log file of
what Googlebot is crawling is easier, or sometimes
really tricky. If you have a website
with millions of pages, and we’re crawling millions
of pages every day, then that’s going to be
really tricky to figure out, or to kind of analyze without
using some kind of specialized tools specifically for that. So I’d say, for the most part,
the average webmaster doesn’t need to analyze their log files. But if you want to dig
into the technical way that Googlebot is
crawling your site, and understand which
parts of your site we’re crawling when, then by
all means go ahead and do that. It’s not always easy. But it’s always an
interesting exercise, I think. MALE SPEAKER: Hi, John. I have a question. JOHN MUELLER: Sure. MALE SPEAKER: Yeah,
my question is about in the News
segment of web search. Basically, what happens when
Google’s picks are used? And at times, what happens
[INAUDIBLE] somewhere from the body, and just
picks something shown there. So the headline is not there,
but something else is there. So why is it happening? JOHN MUELLER: Is
that in Google News, or in the search block that’s
kind of like in the news? MALE SPEAKER: Yes. The web search in
the news section. JOHN MUELLER: OK. I believe we essentially just
use the normal title changes that we usually use
for web search there. So we have a Help Center
article on titles. And essentially, what
we’re trying to do is understand very briefly
what this page is about. And that means that
if we recognize that the title that’s specified
on the page is extremely long, then we have to kind
of rewrite that anyway. If the title
specified on the page is kind of keyword
stuff, where it’s repeating the same kinds of
keywords over and over again, then that’s something
that we’d say we’d like to rewrite in a more
kind of readable way. There are some other
cases also mentioned in the Help Center there. But essentially,
what I’d recommend doing is really making sure
that the title you specify on your site, on your
pages, is as short and as direct as possible, so that
it’s really clear that this is something we could
easily show in Search. It’s not too long. It matches the
content on the page. And it’s something that
tells the user very briefly what this page is about. MALE SPEAKER: Do you have
any specific to– yeah. I’m just giving an example. So this particular one,
let’s say, a movie review. So I mean, that’s title
says this specific, let’s say, [INAUDIBLE]
movie review. That’s my headline of the story. So when did Google
start picking up the headline as their
star cast, so [INAUDIBLE]. So if my news is
about the review, but the news section,
particularly that news answer box showing
that star cast there. So somewhere, I just
feel like this is not relevant for my audience. JOHN MUELLER: So it’s picking
up other parts of your pages? Or– MALE SPEAKER: Yes. It is actually picking
up my star cast. Basically, the actor and actress
names in place of the headline. JOHN MUELLER: I probably
need some examples. So if you can post in
the chat some and URLs that you’re seeing there,
then I can definitely pass that on to the team. But I need to take a look
to see what exactly you’re seeing there, so I
understand what’s happening. MALE SPEAKER: Sure. I’ll do that. JOHN MUELLER: OK. I’ll pass that onto the team
to take a look at to see what we can do there to recognize
the content a bit better. We’re using two websites about
laser engraving machines– one for small businesses,
one for corporations. The context differ. Text and images. But the keywords and
company data are similar. Would a single website perform
better than two separate ones? I think when you’re
looking at two websites and they’re really targeting
different audiences like this, then
that’s essentially a marketing question. It’s not so much of an SEO
question of whether you should have one website
or two websites. For some kind of audiences,
it makes sense to split these. Sometimes it’s just easier
from a technical point of view to maintain one website,
which makes it easier for you, and makes it easier
for our systems to kind of focus everything
on just one website. So in general, if you can
reduce the number of websites, that’s a great thing. With two websites,
I don’t think it’s the case that it’s
completely unmaintainable to have two websites. So that’s something where
two websites are fine. It’s not something where I’d
say from Google’s point of view, you have to do it like this,
or have to do it like that. MALE SPEAKER: Hi, John. Actually, this
question was from me. JOHN MUELLER: Oh, great. MALE SPEAKER: The question
was because all the car manufacturers tend to have
everything on one website. So I won’t talk about
brands here, but just imagine you’re talking about
personal cars and trucks. They put everything
on one single website. And because everybody
knows, OK, this brand– I’m not going to
name anything– is manufacturing a lot
of different sets. So same will be true for us. I mean, maintaining one
website is definitely easier. Totally go along with your view. But the thing is,
we are one company, and the information, the source,
is just one single company. So would that be a problem
from your point of view? JOHN MUELLER: No. I think that would be fine. I mean, if you really want
to separate these two sites, I don’t see a problem with that. I would see a problem if you’re
going past five or 10 websites, and saying, oh, for
each product color, I have a separate website. Then that starts looking like
setting up doorway sites. But if you have two
sites, and you’re saying these are really
different audiences that you’re talking to, these
are two websites with different audiences,
that’s perfectly fine. MALE SPEAKER: Because a
lot of external agencies, they propose that we merge
them into a single one, and– JOHN MUELLER: Sure. I mean you can do that too. MALE SPEAKER: –there’s just one
more specific thing about this. We would then only have
one single home page. So the home page would be
about getting the audiences into the right areas. So for example, you’re a small
business, this is your area. You’re a big corporation,
we recommend this area. And now, we have home pages
with very specific information already. And this helps us
to rank better. Because homepages are
a very influential part of the whole website, I’d say. JOHN MUELLER: Yeah. I mean, essentially,
it’s up to you. You can do it either way. I don’t want to say you need
to do it this way or that way. With two websites, I see
that as completely up to you. That’s not something
where from search, you have to do it like
this or like that. MALE SPEAKER: OK, got it. Thanks for your help. That’s a relief for us. JOHN MUELLER: Great. MALE SPEAKER: Thank you. GARY LEE: Hey, John. I just had a quick question. I posted a link in the chat. I’m not sure if you
can see it there. It was about 10 minutes ago. It was about virtual
office in Hampstead. JOHN MUELLER: Mm-hm. JOSHUA BERG: [INAUDIBLE]. JOHN MUELLER: Oh, OK. As an example where– MALE SPEAKER: [INAUDIBLE]. JOHN MUELLER: OK. I copy these chats out. So that’ll be– let
me just grab that. And yeah. OK. Good. Great. Thanks. That’s always useful. I’ll take a look at
that with the team. Since we’re running towards the
end of time, do any of you guys still have any questions
that we need to get answered? ROBB YOUNG: Can I ask
our usual question, John? JOHN MUELLER: Yeah. ROBB YOUNG: In regards
to the conversation we had on Monday
and the content, I have just a few questions,
but I’ll stick to one for now. The content on the old
domain, because the new domain is going to be an exact
copy, as you know, should the content on the old
site just stay sitting there, or is that going to cause a
problem, that it’s basically the identical to the new one? You said, remove
link connection, so there’s no real connection. But would there be a
content connection as such, that it sees, actually,
this is basically the same one, whatever
issue you’re having? Are they going to see
that that looks the same? JOHN MUELLER: I wouldn’t see
so much of a problem there. I think you mentioned
that you need to keep it up for
legal reasons anyway. ROBB YOUNG: Well, you know– JOHN MUELLER: So I think
that’s generally fine. ROBB YOUNG: I mean,
we can deindex it. We don’t need it spidered. Because as long
as customers have existing links and existing
emails and everything, and visit can read
it, that’s fine. We don’t need Google to see it. So is that the safest? Just essentially, over time,
deindex the whole thing? JOHN MUELLER: You could do that. Sure. Yeah. ROBB YOUNG: Could do that, or? JOHN MUELLER: I don’t see– ROBB YOUNG: I just
don’t want to– JOHN MUELLER: Yeah. I think what might
be confusing for us, if it’s really one to
one copy, if it’s like, exactly the same HTML
files on both sides, then that’s something
where we’d say, well, this is the same thing. We have to pick one of
these as the canonical. But if the old one
like has, I don’t know, the old headings, and
the primary content is essentially the
same, but not like a one to one copy of the
exact same HTML file, then that’s something that
we would treat separately. If it’s exactly the
same HTML file, then that’s something where
we’d say, well, these are essentially the same. We’ll fold them together. ROBB YOUNG: Well,
given how poorly you view the original site,
wouldn’t you by default, then, choose to show the new
one, even though we’ve been struggling for
two years to get you to show the old one anyway? It would be bizarre
for that one to start. JOHN MUELLER: Yeah, yeah. I mean, we would just
treat it as a duplicate. So if you really want to
have that connection broken, then just making sure
that the old site has the old HTML and the new
site has the new HTML, so that we see that
they’re separate pages. ROBB YOUNG: Right. OK. I mean, we can’t use canonicals,
or hrefs or anything, because I don’t want to have an
actual connection between them. So, OK. I mean, we could probably
change the HTML on the old one, actually, and let it re-index
for a couple of weeks. JOHN MUELLER: Sure. Yeah. Exactly. ROBB YOUNG: Because
it’s easier to do that, because we don’t
care about that one. And the new one, we don’t
want to mess with the user. JOHN MUELLER: Yeah. ROBB YOUNG: OK. JOHN MUELLER: OK. ROBB YOUNG: Thanks, John. I’ll think of another
one next time. JOHN MUELLER: Sure. All right. GARY LEE: Hey, John. I’m back again. MALE SPEAKER: [INAUDIBLE]. GARY LEE: Hello? Sorry. Are you still there, John? JOHN MUELLER: Yes. MALE SPEAKER: [INAUDIBLE]. GARY LEE: OK. Yeah, so the link
that I posted earlier, I’m not sure if
you got a chance. I just got cut off
there for some reason. But as you can see
in the results, there’s a very big difference
between the two services, one of them being
a virtual office, and the other one
being a service office. They’re two very, very
different products, yet you’re seeing as a perfect
example in this that over half of the results
that are coming up are related to serviced
office products, rather than virtual office products. And um– JOHN MUELLER: I don’t
really know that market that well, so– GARY LEE: Yeah, well,
I think that’s an issue that specifically
there is going on. There’s also on page two,
there’s websites outranking us that have, clearly even in the
metatags and the index data you can see, it
says zero results. So it’s simply just
like that search results page with no results. And even that’s outranking us. So I kind of feel
like something there is saying that our
site itself just– even though our page is good
quality and a great result, the site itself is still
under some sort of penalty. Or something’s going on. JOHN MUELLER: Let
me just jot down GARY LEE: To be outranked by
a page that has no results, and says in the actual
meta description sort of being pulled
into Google results. It’s pretty poor for us
to not outrank that site. JOHN MUELLER: Yeah. I’d have to take a look
at this with the team. GARY LEE: Yeah. If you could pass that on,
that’d be much appreciated, and see if they can get
back to me or do something. Thanks, John. MALE SPEAKER: Hi John. If you have some– So I just
posted this thing my search query, and under the
search [INAUDIBLE] that particular result. So if you went searching
through this thing. So I have URL in
this chat, itself, so if you can just
go to this one. JOHN MUELLER: OK. Oh, and you mean the Michael– MALE SPEAKER: [INAUDIBLE]. JOHN MUELLER: –Edward
Norton, that kind of stuff. MALE SPEAKER: Yeah. Exactly. Yeah. JOHN MUELLER: All right. Let me just take a quick
screenshot so that we have it. All right. Yeah. That looks like a weird title. That’s probably
something we should be able to do better for. MALE SPEAKER: Yeah. This is basically something
we are seeing of this thing. I mean, like earlier it was
picking the right headline, but now it is picking
up the star casting, so there is something
[INAUDIBLE] us. JOHN MUELLER: Yeah. And that’s like an
element on your pages? Is that right? MALE SPEAKER: Yeah. Yes. [BACKGROUND SPEECH] JOHN MUELLER: Yeah. I can take a look at
that with the team here to see where we’re
picking that up. If you have a thread in the
Help forum, for example, or on Google+, then I can
perhaps give you some feedback there. I don’t know if there will be
something that we can tell you, or that you could change there. But it might be useful. So if you have a
thread somewhere, maybe just copy that
into the event listing, and I can take a look, and see
if I can reply there as well. MALE SPEAKER: Sure. Sure. I’ll do that. JOHN MUELLER: All right. Then let’s take a break here. Lots of questions. Good conversations. Not so focused on Penguin
and links, which is great. Something nice to have,
and some new faces as well. So maybe we can keep that up. But thank you, regulars,
for joining in as well. All right. Then I wish you guys
a great weekend. Thanks again for joining. And hopefully, we’ll
see you guys again in one of the future Hangouts. MALE SPEAKER: Thanks, John. MALE SPEAKER: Thank
you very much. MALE SPEAKER: Thanks, John. Thank you. JOHN MUELLER: Bye, everyone.

3 thoughts on “English Google Webmaster Central office-hours hangout

  1. TechValue Post author

    Hi webmaster team! Techvalue.gr is my blog. Probably you already know there is to much competition between blogs.
    I am going to give you a fresh example to understand my question, lets say a blog has 30 articles about Nexus 6 and another blog has 5 articles about Nexus 6, is that a SEO factor for a site to rank higher in that keyword (nexus 6)?

  2. Mark Marino Post author

    Using rss feeds may get picked up by Google within a few minutes of publishing? Clearly a far better option than basic XML sitemaps, no? I've been himming-and-hawing over converting to rss for a few weeks now (a sporting and events tickets website) as the site adds/drops tons of new pages every day.


Leave a Reply

Your email address will not be published. Required fields are marked *