Tricky Wordpress Changing Permalinks Redirection Question

[Darth]

Currently I don't have categories or anything else in my slugs, posts are just:

www.domain.com/title-of-post/

I am updating my permalink structure to include the category though (actually a category and a sub-category), so it will be like:

www.domain.com/cat1/cat2/title-of-post/

I know that when you are editing a post and you edit the slug, that WP will record that edit and automatically put a 301 redirect for you. However this does not work for changing the permalink structure like I am.

I have over 2,000 posts and have been looking for a plugin to somehow help with this to no avail. There is also no code that I can find (like matching the "title-of-post" part and auto 301'ing based on that).

So right now my best strategy seems to be bulk extracting all my post URLs, separating them into the new categories by group, and then running some sort of text manipulation to create the required .htaccess rules (or to import them into the redirection plugin). This is a lot of work though.

I also forgot to add that out of all those 2,000 posts that have no category in their URLs already, they will all be going into one of 4-5 possible category/sub-category combinations!

Does anyone else have any other ideas?

 

[CCarter]

Does anyone else have any other ideas?

Don’t do this at all. In fact what you are doing is the exact opposite of what most people recommend: you are extending your URL length making it less likely people will share it and diluting it’s marketing value. People have been recommending for years to remove categories from URLs.

But more importantly this is a “no money” change - once this “website redesign” (side note: this is a form of procrastination from marketing - https://www.buildersociety.com/threads/day-14-mental-strength.1294/#post-13533) is done you will not gain a single new dollar as a result of the outcome, in fact you are more likely to lose money.

This will not increase your revenue therefore is a waste of time. It’s hard but you have to be careful to not give yourself work that doesn’t focus on increasing revenue, profits, or reducing costs.

My question to you is what’s the thinking behind attempting this?

 

[Ryuzaki]

I've seen it go both ways in terms of helping or hurting. I use a /cat1/cat2/ structure on a site. It helps in a lot of places but I just did a series of 30+ 301 redirects in the reverse manner of what you're speaking.

In some cases, the cat1 and cat2 will be generic but related terms that can help your SEO efforts, and in some cases they can end up duplicating main keywords in the URL which can hurt or help depending on your optimization, and in other cases they can be generic and unrelated.

In my case, I removed cat1 altogether and a word from cat2, so that the URL became shorter and less distributed in it's emphasis on words that matter. So it might have been something like /animals/big-dogs/the-slug/ which I then turned into /dogs/the-slug/. I'm seeing a boost in traffic doing that on those posts, albeit small. But they're big earners so it should help all things considered.

You would definitely want to do some kind of Regex rule, because slapping 2000 redirects in your .htaccess isn't a great idea, since it'll be parsed every single page load. But how you'd pull that off, I'm not really sure. I wish I was that kind of wizard.

I doubt that all 2000 of those pages has a backlink worth saving. What you could do is put all 2000 into the .htaccess and create some kind of HTML Sitemap page for them to crawl to find all of the redirects quickly. Once you see that they've replaced the redirects (I'd give it two or three weeks to propagate out to all of the rankings) you could remove all of the redirects except the ones with good links.

But I agree with CCarter, not that this will hurt you in the sense that your rankings will tank, but in the sense that it's going to be a waste of time if you go all the way. If you go in and change all of the internal links on the site so there's no redirects, etc... this whole thing could take a while. If I was at 2000 posts needing to be changed like this, I'd chalk it up to "too late, should have thought about it before."

Google doesn't care about the longer URLs and has said so on record many times, but there is a correlation with shorter URLs and higher rankings. Is that because the internet users prefer linking and sharing smaller URLs or because those webmasters are more cognizant of what they're doing and thus market themselves harder and better? We don't know. It's a correlation and nothing more.

There is the consideration of "virtual silos" that I toss around here and there on the forum. Where you'd be designing a less strict sense of silos through the folder hierarchy in the categories and sub-categories, and you can then interlink a bit more carefully within each, plus you can use schema markup on your breadcrumbs to further enforce this relationship.

But I don't see how you're going to go from shorter, less detailed URLs to longer, more detailed with any kind of text matching with Regex. I also don't see it being worth doing if you go all the way and hunt through the site to replace the old URLs. But I can say very confidently that it won't impact your current traffic.

 

[Darth]

Thanks both, basically I have an EMD that has been getting smashed in SERPs since mid-way through last year.

I have decided to move to a non-EMD and at the same time expand the site's content into a wider niche (up a level in the niche category hierarchy if you will - my current entire site niche will become one of several). I am 100% certain this is an absolute necessity. The EMD is now hurting me.

Before Google used to treat the EMD as somewhat of a brand (ie my brand) and I would rank well. Now Google knows it is not a brand and I am getting smashed and even penalised for over-optimisation. See thread I linked. I have done outreach etc for this site, links are not the problem.

TBH I would have had to make this change in a year or so anyway when the next version of my "thing" comes out. Google is just forcing my hand early.

As part of this I want to silo the content, both virtually (easy of course) and by way of directories.

 

[mmm91492]

Regardless of whether or not it's a good move, here's how I would probably do it based off of the info you've provided:

I'd probably just use PHP / a WordPress loop to generate the list of redirects programatically. There may be a way to do it with Regex as mentioned, but it's beyond my limited understanding.

if ($posts->have_posts()) {
  while ($posts->have_posts()) {
    the_post();
    echo $post->slug.', '.$post->cat.'/'.$post->subcat.'/'.$post->slug.'\n';
  }
}

This isn't a working example (you'll have to query for category and subcategory and possibly parse either / both if you have posts tagged with multiple cats/subcats), but in the end you should have a CSV of old URLs and new URLS which you can pretty easily use to generate a list of redirects and batch search/replace commands to update your URLs throughout your database.

 

[Darth]

Regardless of whether or not it's a good move, here's how I would probably do it based off of the info you've provided:

I'd probably just use PHP / a WordPress loop to generate the list of redirects programatically. There may be a way to do it with Regex as mentioned, but it's beyond my limited understanding.

if ($posts->have_posts()) {
  while ($posts->have_posts()) {
    the_post();
    echo $post->slug.', '.$post->cat.'/'.$post->subcat.'/'.$post->slug.'\n';
  }
}

This isn't a working example (you'll have to query for category and subcategory and possibly parse either / both if you have posts tagged with multiple cats/subcats), but in the end you should have a CSV of old URLs and new URLS which you can pretty easily use to generate a list of redirects and batch search/replace commands to update your URLs throughout your database.

Thank you! So I would execute that PHP code within a post in WP (I know how to do that) and the post itself should contain the CSV?

 

[mmm91492]

Right, you could wrap it in a shortcode and stick it on a draft/private page and just view it in your browser to copy/paste the full list. Depending on your PHP skills you could even output it all to a new CSV file on your server.

Again, that code won't work right off the bat ($post->cat / $post->subcat aren't real properties of the $post object, you'll have to query them directly with something like get_the_terms()) but it should point you in the right direction.

 

[Sisu]

I'd do it through .htaccess. This ( https://yoast.com/research/permalink-helper.php ) is a quick converter to go to the /%postname%/ ending from anything else; I used it successfully to get rid of a much longer previous structure on a site with ~500 posts. If you learn .htaccess well enough, you could do it just about any way you want. That said, the last thing I'd want to do is go from a shorter slug to a longer one.

 

[Darth]

I'd do it through .htaccess. This ( https://yoast.com/research/permalink-helper.php ) is a quick converter to go to the /%postname%/ ending from anything else; I used it successfully to get rid of a much longer previous structure on a site with ~500 posts. If you learn .htaccess well enough, you could do it just about any way you want. That said, the last thing I'd want to do is go from a shorter slug to a longer one.

Thanks but that tool won't work for me as I am going from no categories to several different possible categories. There is no easy way to tell what needs to be redirected where (apart from what @mmm91492 posted so far).

Again, that code won't work right off the bat ($post->cat / $post->subcat aren't real properties of the $post object, you'll have to query them directly with something like get_the_terms()) but it should point you in the right direction.

Thanks, TBH I don't know what that means but I will take a good look and try work it out in the next few days

 

[turbin3]

Honestly, this sounds like far more trouble than it's worth. This is coming from someone who has dealt with the extreme end of EMD/PMD as well as Panda suppression, so I feel that pain more than most.

Virtual silos are, by far, the easiest method.

With subdirectory silos, it's easiest to stick to a parent/child layout for the actual category pages, while the posts still sit at the domain route. This is the typical layout on standard Wordpress, for example, unless you actively change it. There are quite a few large and highly successful sites that use this layout just fine. Far easier to manage too.

The second you start trying to manipulate URLs at scale, things get extremely difficult. This is why I usually recommend against a fully directory-silo layout, as it's just baking in the dependencies and potential issues for later on. People still do it, and it can still work, but you really need to have your content structure thoroughly planned as major changes later might be tough.

Another upside, as Ryuzaki mentioned briefly, is posts being at the root helps give greater control over URL slugs.

 

[Darth]

Thanks everyone, I am still mulling all this over. All very helpful suggestions so far.

Just to be clear, with the virtual silos, EVERYTHING is just domain.com/title-of-post? There is no directories in the URL whatsoever?

 

[Ryuzaki]

Just to be clear, with the virtual silos, EVERYTHING is just domain.com/title-of-post? There is no directories in the URL whatsoever?

I use the directory hierarchy to reinforce it, although it's not a must. You can still categorize like you want and the breadcrumbs markup will push the concept at Google, and they'll show that in the SERPs too. Where other sites will have a URL listed under their Title Tag, yours might show:

Animals > Dogs > Big Breeds > Huskies​

In my case, my categories don't go that deep. It's stop at "Dogs", so I'm happy to include that in the URL path. The main thing being that you have to think out all of your categories very early on and commit to it or you end up where you are now with a nearly impossible and risky task ahead of you.

With or without the category and sub-category in the URL, this virtual concept is as good as a strict physical silo at this day and age. Nobody has built a legitimate silo since the mid-to-late 90's anyways. The trends of the internet created way too much interlinking in the navigation, footer, and sidebars to do that now.

But I will say that interlinking contextually with strict relevancy helps a lot and is a part of the virtual silo process. I don't think it hurts to interlink from a non-related page to your main page, although I've seen people claim that it does. But that makes zero sense for Google, since you can't control who creates your backlinks on which pages. Relevancy always helps, but I think irrelevancy helps a tiny bit still. But it's when you link from a highly relevant page (I craft these on purpose) that you see some boosts.

It goes back to the "mini-net" term I always throw around when I say "virtual silo," where you may publish a money page on "Husky Health" and then create 4 pages about "Husky... Diet / Exercise / Illnesses / Grooming" and link those straight back to the main "Health" page and perhaps between each other where appropriate. So you basically create a PBN on your site for specific pages. And they can extend off your site to guest posts too. But it's all about relevancy with these as part of the virtual silo.

The idea is to force the topical relation in a technical fashion as best as you can. That includes URL directories if possible, breadcrumb markup and breadcrumb links, interlinking contextually, anchor texts used when interlinking, on-page optimization, etc.

The point being that Google doesn't need our help, where we build strict silos, to understand topical relation anymore. And even if they did, not a single person is doing it in the way it was first designed. Categorizing and interlinking contextually is fine and dandy but we're bleeding out of our silo in a million places on every page these days.

At this point, it's like Google is the teacher and we're the student. We don't have to do anything extra to help Google grade our homework beyond the basics of on-page these days. But there are some things we can do, like writing with a black pen instead of a pencil, or typing our essays with proper margins and spacing, to get them to be willing to give us a higher grade subconsciously. It's like asking for bonus points while nobody else is doing it. We're all getting an A just for turning in our paper, but me and you will get an A+ for doing these extra things.

 

[Darth]

Thanks, I am getting closer to a final decision.

I have decided to drop the sub category idea at least. This is based on comments above plus I realised that you can't have two subcats with the same name. Eg "News > Cars > Posts" and "Guides > Cars > Posts"

So I am now thinking either no directories whatsoever (everything virtual) or one level of directory. Probably the former because redirecting all 2K posts from my exisiting setup is just too hard/risky.

Breadcrumb markup and breadcrumb links, interlinking contextually, anchor texts used when interlinking, on-page optimization, etc. are also already in place. My markup is on point especially compared to my competitors.

 

[Darth]

Update

I made the move to the new domain about 2 weeks ago

For the first 4-7 days my original domain stayed in the SERPs at about the same ranks. The next 3-7 days the new domain replaced it in the SERPs at the same positions roughly.

Now approx 2 weeks after the original domain change I am dropping out of the SERPs completely. The old domain has disappeared (normal) and the new domain has dropped from top 10 on the big queries (usual positions) to positions 50-80. Traffic has been smashed of course.

301 redirects (quadruple checked) and WMT change of address etc are in place. The new domain is fully indexed with the content I moved over and has been since shortly after the move.

In terms of the URL structure this whole thread was originally about, I kept it as is but removed a directory/category from the URL of my highest performing pages. Eg they went from olddomain.com/dir/title to newdomain.com/new-title (with 301s in place).

The maximum number of 301 redirects that any URL on the old domain has to its proper address on the new domain is two.

The old domain was an EMD, the new domain is a partial/slight EMD.

Holding on grimly but this is tough! My current situation kinda reminds me of what happened to @animalstyle here https://www.buildersociety.com/thre...rity-expansion-project.2753/page-3#post-32820

Pinging @Ryuzaki @turbin3 as you guys had some good advice earlier

 

[Ryuzaki]

You likely lost some ranking signals when you went from EMD to non-EMD.

Also, did you drop out some kind of keyword when you removed the /directory/ part of the URL?

We saw something similar happen with another member who wanted to expand from micro-site to broader authority site. We never realized he had an EMD until after the fact and we started asking questions. He reverted the change rather than climb back uphill but I now know that he's largely going back to the initial 301 situation. Take that how you will but even after seeing a hit and reverting the changes, he still chose to go back, meaning he only lost even more time.

I don't think it's unexpected to see some bouncing. You changed quite a bit in the URL and lost an EMD bonus. It's a big on-page change. It might bounce back after some period of time (can be days or up to 90 days). You might even have to make up some ground there ultimately but, really it exposes how much you were leaning on the EMD bonus. You'll likely find it was stopping you from optimizing as hard and that you may be able to make up some of the gap with some on-page testing on new content.

Also, you'll have a wider base of areas to create content in and get links to, which will help make up the gap and then keep you climbing.

I have decided to move to a non-EMD and at the same time expand the site's content into a wider niche [...] I am 100% certain this is an absolute necessity. The EMD is now hurting me.

Remember this quote if you start feeling like straying from the target. You're trading a bounce for the possibility of having Google recognize you as a brand in order to have a much greater long-term play on your hands.

Also, try this site out: Redirect-Checker.org and make sure you don't have any crazy redirect chains going on.

I've seen cases of people doing things like:

http://olddomain.com --> 301
http://www.olddomain.com --> 301
https://www.olddomain.com --> 301
http://newdomain.com --> 301
http://www.newdomain.com --> 301
https://www.newdomain.com --> 200

You need to minimize that as much as possible while retaining old redirects too. For instance, if you built a majority of your links to the old site on HTTP and then went HTTPS, you need to maintain that 301 before adding the 301 for the old site to the new site. You don't want to lose all of your old links by overlooking things like that.

But when you finally make the jump to the new domain, make sure you aren't doing the non-www to www and non-https to https electric boogaloo. Aim exactly where you mean to aim and get there in as few steps as possible. Google will give up after too many leaps. 3 maximum is what I suggest.

 

[turbin3]

Ryuzaki is on point with the things I'd be looking at.

Also, depending on the volume of high performing pages you're talking about, there may be another option worth considering. I'm on the fence about this, but thought I'd mention it.

If the subdirectory you removed was in fact one of the big keywords, and if it's not a huge volume of high performing pages, one option might be changing the slugs on those to add that keyword back in. That way you're not reverting to the subdirectory for everything, and maybe just throwing it back in at the beginning of the slug. This may or may not be a good idea, depending on your specific case. Might be a stopgap to ease you through the transition.

 

[Darth]

many thanks both for the speedy responses!

Also, did you drop out some kind of keyword when you removed the /directory/ part of the URL?

I did, but I added it back into the slug. Eg from /word/title to /word-title/

I don't think it's unexpected to see some bouncing. You changed quite a bit in the URL and lost an EMD bonus. It's a big on-page change. It might bounce back after some period of time (can be days or up to 90 days). You might even have to make up some ground there ultimately but, really it exposes how much you were leaning on the EMD bonus.

I would be speechless if losing EMD bonus caused this. There is a site ranking in the top 3 for some terms that literally copied and pasted my pages (seriously) for the same terms (a few years ago, and I have since updated the pages so they are no longer the same) that is ranking in the top 3. It's ludicrous! Ahrefs and SEMRush turn up no amazing links either. My links are far superior as is my on-page SEO. That's why I thought the EMD was a big part of causing these issues (over-optimisation). I simply couldn't understand how those sites rank higher (maybe PBNs I guess) but man their content is crap compared to mine).

You need to minimize that as much as possible while retaining old redirects too. For instance, if you built a majority of your links to the old site on HTTP and then went HTTPS, you need to maintain that 301 before adding the 301 for the old site to the new site. You don't want to lose all of your old links by overlooking things like that.

But when you finally make the jump to the new domain, make sure you aren't doing the non-www to www and non-https to https electric boogaloo. Aim exactly where you mean to aim and get there in as few steps as possible. Google will give up after too many leaps. 3 maximum is what I suggest.

Thanks, I double checked again and am 100% covered. Most are 1 redirect, some are 2 (where I removed the category and changed the slug). Both www, non-www and HTTPs and HTTP are all sorted

If the subdirectory you removed was in fact one of the big keywords, and if it's not a huge volume of high performing pages, one option might be changing the slugs on those to add that keyword back in. That way you're not reverting to the subdirectory for everything, and maybe just throwing it back in at the beginning of the slug. This may or may not be a good idea, depending on your specific case. Might be a stopgap to ease you through the transition.

I moved from the word from the directory to the slug as you suggested when I did all this

To sum it all up I really can't see any reason how I could have moved down to #90 for some of these terms. My backlink profile is superior, my on-page is tight (and hasn't changed from the old domain). All I did was change domain and remove a category (and put the category word in the slug). I updated all my internal links to point to the new domain. There is no trace of the old domain on my new site whatsoever.

Edit: One other thing I did was change theme. I was using the old 2014 standard WP theme (customised) and now have a paid theme as part of this rebranding. All my micro and rich data mark ups etc I moved over too though. I also cut down the amount of banner ads in my content.

Surely this just has to be some sort of period where I just wait while Google figures all this out. Most of my backlinks are to the HTTP version of the old domain, then I moved that to HTTPs about 6 months ago (ZERO ranking changes) and now I have moved to a new HTTPs domain. Maybe they need some more time to process everything (notwithstanding all my 301s are spot on). Although that said it did look like they were getting it (see my post above) a few days after the changes were implemented (new domain appearing in previous ranks) but then everything just went out the window.

 

[Darth]

Pinging @CCarter also if you have any advice? I've seen your (and everyone else's comments in here too) and to respond to some of the things you said there (in addition to everything above) and others:

• My keywords are in the URL (primarily slug), H1 and title - all three covered
• I haven't used the URL remover tool (seems extreme!!)
• All internal linking within article bodies is the same (and changed to point to the new domain and new slugs where relevant). However, on the old site internal links to my top pages were more prominent (I changed the theme of the site and menu structure a bit). This could be another cause for my dip

Also @CCarter in response to your last post in this thread (don't do it) you'll see I didn't. I am 100% open to following the advice of the gurus here.

If the subdirectory you removed was in fact one of the big keywords, and if it's not a huge volume of high performing pages, one option might be changing the slugs on those to add that keyword back in. That way you're not reverting to the subdirectory for everything, and maybe just throwing it back in at the beginning of the slug. This may or may not be a good idea, depending on your specific case. Might be a stopgap to ease you through the transition.

I did this and added in fresh 301s (still checking to ensure any redirects are 1 or 2 levels max)

One other thing - I removed a ton of text from my HP which was a custom page in WP and is now the default "last 10 posts" in WP. This was in part due to the new theme, but also the direction I want to take the site under the new domain (not so focused on the "key money pages" topics, branching out), but I could re-add that page as the HP if I wanted. It prominently featured links to the key money pages that have lost considerable rankings.

My current thoughts

Based on everything discussed it seems my problems could be due to:

1. Losing the EMD (hard to believe/understand when I was convinced the EMD was part in due to the initial drop on the old domain)
2. New WP theme
3. New menu structure
4. Removal of custom HP that included certain key text and prominent links in the body to the money pages that have lost the ranks
5. Just gotta wait longer (but weird seeing that the ranks moved over to the new domain and pages perfectly for a few days and since tanked?)

Or a combo of the above.

 

[Ryuzaki]

Did you see a site-wide drop or just on the posts that lost the page rank flow from losing links on the homepage and a less advantageous menu structure?

 

[CCarter]

When I wrote my reply you didn't say anything about a brand new domain. You also didn't say anything about a brand new theme. There is a ton of new information now, but the short answer is - you changed WAY TOO MANY things in a short period of time and are screwed.

The 301s expansion was a bad idea alone, but changing the theme of the website along with having a new domain that doesn't have the keywords... Where do I begin?

To give you an analogy - Imagine if Apple changed it's name to Orange. A long history and tons of customer loyalty would go out the window. In your case Google is the customer - trying to evaluate whether their visitors should come to your website and if you are worthy of traffic from them. Changing a brand - no problem, it is a problem but not mega. Changing a theme - that is somewhat expected during re-branding, as well as doing 301 redirects. However you are expecting to bounce back immediately?

That doesn't happen - when mega brand update their themes they take a hit in traffic. When mega brands change their brand they take a hit in traffic. When mega brands change from HTTP to HTTPS the take a hit in traffic. When mega brands change URL - they take a hit in traffic. So if mega brands take a hit, your smaller website is definitely going to take a hit.

It's simple, you are going to have a wait it out and start blogging (adding content) that is relevant. Go after long-tail keywords in the content and traffic leak your target audience.

There is literally nothing else you can do and if you are planning on doing anything else drastic - STOP. You can't make major changes and expect to recover over night. You have to wait for the recovery period to process.

 

[Steve]

Its likely to be down to the homepage change.
From what i can gather from your post, the homepage used to be a big chunk of unique text full of keyword rich contextual links to your main money pages, and you replaced it with a list of excerpts from your blog.
The homepage is also likely to be the most linked to page on the website.
You removed those keyword rich contextual links from the most powerful page on the website, and then the linked pages dropped.

 

[Darth]

Did you see a site-wide drop or just on the posts that lost the page rank flow from losing links on the homepage and a less advantageous menu structure?

It was mainly the posts that lost the page rank flow I think

It's simple, you are going to have a wait it out and start blogging (adding content) that is relevant. Go after long-tail keywords in the content and traffic leak your target audience.

So when you say I am screwed, I am not screwed permanently? I can recover from this if I wait long enough? NP as I am patient and already blog regularly.

From what i can gather from your post, the homepage used to be a big chunk of unique text full of keyword rich contextual links to your main money pages, and you replaced it with a list of excerpts from your blog.

Yep, pretty much this

From my list of things above I am leaning to 3,4,5 at this stage.

 

[Darth]

And I'm more or less back

 

[Jared]

Much ado about nothing, eh?

 

[Darth]

Much ado about nothing, eh?

Haha, no I think this thread is incredibly valuable. Thank you again for everyone's help. And this is also just one day (and they did rank this high previously on the old domain, only to fall) so I haven't quite "won" yet. But it's promising.

I am still going to carefully consider my 3,4,5 of my list above and have already reached out to the theme maker for advice on how to edit the HP so it's more a hybrid between old and new.