All Low add-ons are now owned by EEHarbor. Read the blog post.

Blog

Nesting tags and performance in EE

23 September 2010 13 comments

Not too long ago, I tweeted:

Pro tip: don’t ever, ever nest a channel:entries tag inside another tag that loops. Srsly. Don’t. #eecms

...which sparked some response. The main reason for not nesting ExpressionEngine’s channel:entries tags is performance. Let me explain with an example.

Category archives

EE’s category archives tag is limited. To overcome this, some people use a channel:entries tag inside a channel:categories tag. Something like this:

{exp:channel:categories channel="news" style="linear" disable="category_fields"}
  <h2>{category_name}</h2>
  {exp:channel:entries channel="news" category="{category_id}" dynamic="no" disable="member_data|pagination|category_fields"}
    <h3>{title}</h3>
    {news_body}
  {/exp:channel:entries}
{/exp:channel:categories}

Now, this will work. However, given enough content, this will slow your site down considerably. The number of queries that will be executed to generate the output is not fixed. Instead, a new channel:entries tag will be generated for each category found, which will result in an unknown number of queries. Dozens at least, hundreds if you have lots of content.

I did a test where I pasted the above code in an empty template, turned on the profiler and checked the number of queries used. With 4 categories and 6 entries, it took 41 queries to generate the page. Then I added a new entry in a new category and reloaded the page. This time it took 45 queries. Just imagine what would happen if there are 20 categories with 200 entries. And that’s not even a lot.

Fix it

To fix this potential problem, we need to make sure the amount of queries needed to generate the page is fixed. We can achieve this by un-nesting the two tags and using PHP on Output. The code is a bit more complicated, but it will improve performance dramatically. Here’s the code in full. I’ll explain each bit below.

<?php
  $entries = array();

  {exp:channel:entries channel="news" dynamic="no" disable="member_data|pagination|category_fields"}
    $entry =<<<EOE
      <h3>{title}</h3>
      {news_body}
EOE;
    {categories}
      $entries[{category_id}][] = $entry;
    {/categories}
  {/exp:channel:entries}
?>

{exp:channel:categories channel="news" style="linear" disable="category_fields"}
  <h2>{category_name}</h2>
  <?php if (isset($entries[{category_id}])): ?>
    <?=implode("\n", $entries[{category_id}])?>
  <?php endif; ?>
{/exp:channel:categories}

First, we’ll define an array ($entries). Using the channel:entries tag—without a category= parameter—we’ll fill that array with entries grouped by category. After putting the template chunk for an entry in the $entry variable, we’ll loop through each related category and add the entry to $entries. What we end up with, is a nested array with all the entries in the right order, grouped by category_id.

To output the entries, we’ll simply use the channel:categories tag to display each category. We then check if there are entries set in the $entries array for that category_id. If there are, we’ll echo them to the page after implode-ing them to a string. All done.

Performance FTW

So, exactly how many queries did we save using this method? I reloaded the page using the above code, with 5 categories and 7 entries: 29 queries. That’s right, 16 less than the nested method. How many queries would it take if there are 20 categories with 200 entries? 29.

Comments

  1. John D Wells 23 September 2010 at 14:07

    That is just absurdly, ingeniously clever.

    I’ve gone through your “PHP in Templates: Pain or Pleasure?” a few times and never really had much of an “ah-hah” moment, but this did it.

    Thanks!

  2. Joe 23 September 2010 at 17:02

    I usually prefer just to do the work in php above instead of fudging ee loops to get what you need. Makes it less messy and gives optimum performance. Obviously it would require php knowledge, but then you’d need some to do your method anyway! Just my opinion :-)

  3. Ty Wangsness 23 September 2010 at 19:28

    I’ve consulted on sites that nested query-generating tags within each other multiple times and were using 500+ queries to generate the page.  Once I was done optimizing them they were back under 40 and executing in a quarter of the time.  It’s very easy to do this without realizing it, especially with embedded templates inside a channel:entries tag.

  4. Low 24 September 2010 at 07:23

    Joe, the reason I’m using the native EE tags is simple: I don’t want to re-invent the wheel. The channel:entries tag allows you to define with great flexibility which entries you want to display and it takes care of the typography. Doing everything in PHP would result in even more code and less maintainable templates. You get the best of both worlds like this.

  5. Dylan 24 September 2010 at 08:47

    Thanks for sharing this technique as well as your great slides from EECI.

    I’m refactoring an index template that takes an image from the first matrix row for a series of posts. I’ve created a query and saved the image paths in an array indexed with the entry IDs. Then when I’m outputting the entries with the channel:entries tag I’m struggling to get the imgsizer plugin to use the array for the image paths. I know it’s due to the parsing order… any tips would be awesome.

  6. Joe 24 September 2010 at 09:37

    I’d have to disagree, not just because it would actually be done in less lines, but also because keeping php/processing separate from output is far more manageable, hell you could even just require a functions file and keep the template very clean.

    The query needed to grab what you’d need in your example is very simple, and would increase performance. It’s just my 2cents worth anyway, of course every ee developer works differently. Maybe I’m too used to working on very large scale ee sites and need things as clean and fast as possible.

  7. Low 24 September 2010 at 10:01

    Sure, for this example it might work. But, like I said earlier, you are going to re-invent the wheel like that.

    First off, you need to write an SQL query, which is easy enough for the above example, but gets trickier if you’re going to work with more/different parameters. Using the EE tags gives you the ease of specifying the entries without having to write up a huge query, looking up all field_id_x’s, etc.

    Then there’s the variable parsing and custom fields. The {title} is parsed using the xhtml-lite typography, the body field with (for example) Textile. You’d have to take care of that as well, if you’re not using the EE tags. Again, not that difficult for a seasoned developer. But what if you’re also displaying a relationship field? Or a Matrix field? Would you want to do that with PHP, too? I know I wouldn’t.

    And if it’s keeping the templates clean that’s important, then you’re probably even better off taking the PHP out of the template and putting it in a custom add-on.

  8. Joe 24 September 2010 at 10:13

    I understand what you’re saying, but for the most part it’s really pretty simple SQL, it really wouldn’t be a huge query when you’re just building an array for reference and not ouput.

    For variable parsing / custom fields id leave EE to do that, as it’s more output based. But in terms of telling the EE loops _what_ entries to output, and in what order, normally Id get php to do that, pass set entry id’s into the EE loop which keeps it all nice and clean.

    I hope that makes sense, I’m not trying to undermine what your methodology at all - just wanted to share my way and why :-)

  9. Hans 14 April 2011 at 07:40

    Thanks a lot for this solution. I like your thoughts about not re-invent the wheel!

  10. Jake Mauer 14 April 2011 at 22:38

    Could you explain what “EOE” is?

  11. Low 15 April 2011 at 05:58

    Jake, that’s the identifier for the heredoc string notation and in this case simply an acronym for ‘End Of Entry’.

  12. Scott 26 May 2011 at 04:31

    Thanks so much for this code! It has been a lifesaver for me. One question though… how would this work with parent/child categories? I’ve tried it as-is and it lists everything properly except that it includes the child entries in with the parent instead of just the child. I’m trying to avoid having the entry in two places. Example:

    Parent Category
    - Entry 1
    - Entry 2
      Child Category
      - Entry 2

  13. Low 31 May 2011 at 21:03

    Scott, you could try and set the Auto-Assign Category Parents setting in Global Channel Preferences to No. That way you’d assign an entry only to a child category, not to the parent.

Commenting is not available in this channel entry.