Portions of this page come from comments made by User:Jamesday and myself on Template talk:Sisterproject. -- Netoholic @ 19:05, 2005 Feb 4 (UTC)

Doesn't most of this apply to any popular template? In other words, if Wikipedia:WikiProject Stub sorting had not begun, or were to be called off, wouldn't this apply to the original {{stub}} template as well? — Itai (f&t) 14:41, 7 Feb 2005 (UTC)

Certainly, the potential for vandalism exists, and the template could be protected for those reasons. The server load problem still partially exists because of the use of the category in that template. Meta-templates are worse though. -- Netoholic @ 16:00, 2005 Feb 7 (UTC)

Could be. Basically, what I'm saying is this: are we sure this is not part of a greater problem? Seems that the problem is not meta-templates per se, but rather any template that is used on more than n pages. — Itai (f&t) 16:55, 7 Feb 2005 (UTC)

It seems to me that a large part of the problem stems from the different treatment given to anonymous users, who are served by massive caches which must be invalidated every time an edit is made. Registered users do not reap the benefit of these caches, because of the way the software implements the "logged-in-ness" of a given user.

Maybe some effort should be applied to making the software more efficient for registered users, if necessary at the expense of worse performance for anonymous users, which would encourage more people to log in.

I'm not expert enough in the whys and wherefores of XHTML/CSS to understand quite why anonymous users should be able to use caching—and logged-in users cannot—when I thought the real difference is in the CSS "wrapper" which should be independent of the "real content".

Enlighten me, someone, please!

--Phil | Talk 11:35, Feb 8, 2005 (UTC)

Happy to.:) When a page has been removed from caches, the first view of the page causes several steps significant to this discussion:

• Each item in the base page portion is requested from the database (images and CSS aren't in the main part). The page you edit, each templage, each template included in the template and so on. Two templates, to database records to be retrived. One template on its own, one read, one template including another, two. Plus the one for the base page.
• Once that and the rest of what is called the parsing is done, the page is saved in the parser cache. That's kept in RAM in memcached, subject to a capacity limit, currently 12 memcached programs running, ach holding 280MB for a total of 3.3GB of cache (a bit is used for other things but this is most of it). I see that 8 more are currently disabled - probably a temporary problem. In a few weeks I expect it to be about 12-15GB for this.
• Finally the skin is applied and the page is passed on to the Squids, which cache it for all who aren't logged in (will only be useful if it's the normal skin) and send it on to the person who originally requested it.
• Whenever any part of a page is changed, be it the page itself or a template or image used in it, the page is marked as changed and will be regenerated next time it is requested. Both the Squid and parser caches have it removed. Necessary so people see the correct version.

The Squids, because of the limitations in the way they can work, with much less work per page, are inherently the fastest way of serving the pages and just 4 machines can serve some 75% of all hits to the site. But they are restricted in what they can serve. Next step is using the parser cache via the apache web servers. That allows all of the user settings for logged in people but uses more web server CPU so it's less efficient. But can't yet be avoided (we're working on it - we want to serve as much as possible from cache).

We could switch everyone to using the apaches but that would be far less efficient and would require something like 4-5 times as many apaches as we have today, far more than the 4 machines gained by not using them as squids.

So, logged in people aren't being discriminated against. They also get caching. But the need to support user preferences does mean that they can't be served as efficiently as those who aren't logged in. Jamesday 08:09, 13 Feb 2005 (UTC)

I don't understand why you need to reparse the page in order to support user preferences. There are only two preferences that affect layout (3 options for dates and 6 options for math). So why not include all 9 options in the document, using CSS and JavaScript to select which one gets presented in the browser? Then the same cached page would work for all users, logged in or not. Gdr 16:07, 2005 Mar 22 (UTC)

Just in case anybody was not clear on the subject - this policy is, at the moment, being considered. It has not been endorsed by the Wikipedia community. No decisions should be based on it. — Itai (f&t) 22:29, 8 Feb 2005 (UTC)

This is not intended to be "policy". The Be bold guideline isn't "policy" either, but that doesn't mean we should ignore its wise guidance. -- Netoholic @ 17:25, 2005 Feb 9 (UTC)

The wiseness of this non-policy is being debated as well. In other words: don't point people, through edits summaries or directly, to this page as an explanation for actions without pointing out that this is not an official policy, merely the opinion of some Wikipedians. — Itai (f&t) 17:38, 9 Feb 2005 (UTC)

I see no debate refuting the points made in this page. Your argument on other talk pages seems to be about templates in general, but that doesn't mean the assertions of this page are incorrect. Beyond simple vandalism, meta-templates have other technical problems, which are documented here. -- Netoholic @ 18:22, 2005 Feb 9 (UTC)

Not policy but Netoholic is right in suggesting that templates in general and more so, templates within templates, increase server load . Better not to use meta templates if relatively few templates are involved and there's much concern for efficiency and page loading times. Jamesday 08:09, 13 Feb 2005 (UTC)

Can you explain a little more on why templates in general are a problem, and how much of a problem they are? BlankVerse ∅ 13:08, 13 Feb 2005 (UTC)

As I mentioned above, I'm not sure most of the content of this page is directly related to meta-templates. How about moving everything that is not directly related to, for instance, Wikipedia:Problems with frequently used templates (along with a note saying that meta-templates often fall under this criterion), and keep on this page (which I would rather have moved to Wikipedia:Problems with meta-templates, but this really isn't important) only what relates to meta-templates (along with a note pointing out that meta-templates are often frequently used). — Itai (f&t) 19:18, 10 Feb 2005 (UTC)

I've tried to assist by highlighting some of the general "big template" issues and the ones whcih are specific to templates wihtin templates. Basically, templates within templates because they are likely to affect more pages, do have a larger effect than is initially apparent, mostly because of that lag issue, but also because of the extra base load for the database servers. Base load we can buy more machines for, that lag is a far tougher problem. You cna probably extract the generic template bits for a page on that. Jamesday 08:09, 13 Feb 2005 (UTC)

Whatever is true of regular templates is twice as true for meta-templates. Please don't split out sections of this page, and don't rename it. -- Netoholic @ 20:55, 2005 Feb 10 (UTC)

I've had no intention of taking any such actions unilaterally. However, I still maintain that this page is misleadingly named. This is like a Ford advertisement saying: "you shouldn't buy Hondas because Hondas can't fly." True, but the same can be said of all other automobiles. Wouldn't it be better to have a site-wide policy regarding all frequently used templates (say, those having over 10,000 occurrences), and keep this page for problems restricted to meta-templates? (Along with a note saying that some meta-templates also have the misfortune of being frequently used, and thus pose the problems mentioned on the newly created page.) — Itai (f&t) 21:58, 10 Feb 2005 (UTC)

If you like, I'm sure that when template parameters were first considered, there was a lot of discussion regarding what's wrong with parameters. You can add the arguments against allowing parameters to this page. (Meta-templates generally use parameters.) In fact, if you wish to make this page longer still, you can even include bits about what's wrong with wikis in general - which applies, admittedly, to meta-templates as well. — Itai (f&t) 21:20, 11 Feb 2005 (UTC)

I've no significant load concerns about parameters in templates. The parsing cost is minimal compared to the extra database reads of templates or templates within templates. People may have stylistic and taste objections but that's a different thing. Jamesday 08:09, 13 Feb 2005 (UTC)

Itai, stop trolling. This page presents many of the major issues of meta-templates. Vandalism is only one, but it's impact is felt even harder than when using one template. For example, if I have meta-template that governs 20 other templates used on 100 pages each (2000 pages affected), it is more damaging to vandalise the one meta-template than it would be to vandalize a single template used on those 2000 pages. -- Netoholic @ 21:48, 2005 Feb 11 (UTC)

Netoholic, stop murdering infants. (You're not, but neither am I trolling.) If using a meta-template-generated templates for these 2000 pages is indeed worse than using a single template, list the reasons for this on this page. You've still to provide one good reason why it wouldn't make more sense to list the problems with all frequently used templates on one page, and keep this page, which has "meta-templates" in its title, for problems specific to meta-templates. (Including, for instance, the reasons for your above statement.) — Itai (f&t) 02:08, 12 Feb 2005 (UTC)

Each meta-template doubles the number of template records which must be retrieved from the database when the page is being built, slowing down the page view and adding database server load. That's true regardless of caching questions. If a meta-template is used in 5,000 other templates it may be worth having anyway because of the amount of work it saves (though it's probably debatable whether it is really wanted by all 5,000). If it's used in 10, twice the number of template retrievals seems like a high cost to pay compared to the human work saved. We can solicit donations to buy more hardware to do that work, but it's nice to try to be efficient as well. For this reason, meta-templates are a special, more costly, case of the general big template situation and do merit some extra consideration. Jamesday 08:09, 13 Feb 2005 (UTC)

What if we quit using those templates that are currently being used as meta-templates as meta-templates, but just use them only as a common template design, and then create the "daughter" templates using {{subst:meta-template}}? That should get rid of the double read.

From your description, it also sounds like we should be trying to keep down the number of templates per page. I've seen a few pages that have been plastered with an attention template, a dispute template, and a stub template. Should there be a policy that in general there should only be one (or two?) templates/page?

Although I haven't seen you address the issue, I am guessing that there may also a problem with some of the very large templates that are sometimes being created (compared to small templates)? Also: How easy would it be to get a list of the most commonly used templates with an estimate of the number of pages they are being used on? BlankVerse ∅ 09:47, 13 Feb 2005 (UTC)

As I discussed at Wikipedia:Templates for deletion#Template:us-geo-stub, using lists, because of the large number of entries, and the large amount of human maintenance that would be required, is an unworkable alternative for trying to deal with stub articles. The topic stubs were created because the Category:stub had grown so huge that it had to be disabled because of the hit on system performance. The Category:substubs was also growing to an unmanageable size. Regrouping the stubs into smaller stub categories allows individuals who are interesting in those topics, plus the Regional notice boards and Wikiprojects, to find and expand those stubs into larger articles. As far as I can see, the best solution for handling the topic stubs may be to quit use the metastubs except that the text for the metastubs is saved somewhere so that they can be used as text templates for creating topic stubs with a standardized format. BlankVerse 09:37, 11 Feb 2005 (UTC)

Can I ask this... why are unsorted stubs such a problem? Take us-geo-stub, for example - I can guarantee there is noone sitting around saying "Gee, I think I'll see what random U.S. locations are stubs, and improve one". Let's leave them to be happy little stubs, sitting out there doing no harm and not being referenced, until someone actually interested in that subject comes along and improves it. Sorting stubs is busywork. For any specific topic area, wouldn't it make more sense to just make a list of the needed articles, and prioritize them for work? I'm not saying you must maintain a list of every related stub, but why go out searching for them just to categorize them? The problem is, the stub message should never have had the Category:Stub added to it. We only needed a simple text to encourage people to come in and edit. -- Netoholic @ 15:43, 2005 Feb 11 (UTC)

1) I have already explained to you in the Wikipedia:Templates for deletion#Template:us-geo-stub discussion that I have used the Template:Japan-stub/Category:Japan-related stubs combination to look for articles on Japanese topics to un-stub. I've heard of other Wikipedians doing similiar things for other subject areas. So yes, there are people who are using the topic stubs to help un-stub articles.

2) Leaving the articles as just stubs will guarantee that many of them will NEVER be upgraded unless someone comes along interested in that specific article rather than that subject area.

3) Having a huge number of articles with template:stub has been a problem because category:stub because so huge that it was big performance hit and so the stub category had to be removed from that template. That means that the only way to find those Wikipedia articles with the stub template but are not in the stub category is to do a Google site search.

4) Lists would be a HUGE amount of busy work if there were to be used to keeping track of stubs. BlankVerse ∅ 13:04, 13 Feb 2005 (UTC)

I can guarantee there is noone sitting around saying "Gee, I think I'll see what random U.S. locations are stubs, and improve one". Can you now? I don't spend a whole lot of time looking at the geo-stubs, but I do check in occasionally to see if there are any that I recognize and can help out. However, while I find the geo-stubs marginally useful, I'm really not convinced about the entire elaborate new stub sorting apparatus. Seems unnecessarily complex to me, but then there may be others who, like me, check for stubs in their areas of interest from time to time. So count me as decidedly ambivalent. I can see how a *simple* stub sorting method is superior to making lists (it is trivially easy for a random editor to add a stub tag to an article, while adding the article to an appropriate list is considerably more involved). older≠wiser]] 16:03, Feb 11, 2005 (UTC)

I agree with you. That guarantee was pretty much a blatant assumption; all editors don't (and shouldn't) edit the same way. That is exactly what people are doing. I often pick a stub topic and go through it, copyedit, google it, get out a book I have on the subject. It's pretty difficult to find articles that really need development otherwise. Randomly clicking hoping to find an article in the subject I'm interested that really needs it is a waste of my limited time. --Sketchee 16:05, Feb 28, 2005 (UTC)

Personally, not speaking as the developer most responsible for database issues for this comment, I'd like to see stubs in both alphabetic and subject categories (with alphabetic grouped by the first 3 letters of the page name). There are people who like to work on specific topic areas and also are people who like to just work on random stubs. (shrug) Easy enough to help both groups. There's no extra cost for this in the individual page views. Alphabetic grouping for the full stub list to keep the category sizes manageable. Jamesday 08:09, 13 Feb 2005 (UTC)

Now as a developer: large categories were a significant load problem. They are still an issue but less so and may be a smaller factor with MediaWiki 1.5, whenever that arrives. For load reasons it's nice to keep the total number of items in a category in the few thousand range. For general guidance, we use 500 for the all pages lists and such because 500 is currently about the maximum normal comfort level. More than that starts to become an issue and in the thousands I start to notice them showing up as unusually slow queries which are delaying others. Try to think how big the category will eventually be and segment it in some way with a target in the 500 or fewer members range if you're trying to avoid load issues. In November or December 2004 I requested an emergency change in how categories were handled (hard limit to 500 members). I also acted as a developer to temporarily protect one template at a version without a category or image in it for load reasons. Another change made it possible to remove that limit and protection, though it's an imperfect solution at present - just good enough to dodge a hard limit. There's a change coming in MediaWiki 1.5 which may raise the 500 number by reducing the cost a bit. Jamesday 08:09, 13 Feb 2005 (UTC)

Can you tell us how many Wikipedia articles currently have the stub and substub templates? Also: which categories on the Wikipedia currently have the most number of articles? Finally: Are images in templates a performance issue? BlankVerse ∅ 13:04, 13 Feb 2005 (UTC)

Netoholic: Your description was overblown as well as incorrect. I still left your basic points intact. Without the exaggerations that were in your original version, people will actually probably believe your position more. BlankVerse ∅ 17:36, 12 Feb 2005 (UTC)

There are no exaggerations. The wording may be "dramatic", but this page serves to describe a strong position against the use of these meta-templates. -- Netoholic @ 01:00, 2005 Feb 13 (UTC)

From your description in the Scope creep section:

• " literally hundreds": with roughly 250 stub templates in Category:Stub categories, it should have said a few hundred.

• Baring mind, if we have 250 stub templates for roughly 475k articles, how many do you think we'll have when the Wikipedia reaches 1 million articles? Certainly not 250 stub templates. Maybe 500, maybe more. We might have as many stub templates and stub categories as we do have categories now. -- AllyUnion (talk) 06:31, 16 Feb 2005 (UTC)

• "many of which have been created on the spur of the moment": False. If you had check the various pages for the Wikipedia:WikiProject Stub sorting, you would have seen that the new topic stubs were being created with specific needs in mind. For example, many of them are being created to be associated with different WikiProjects or Regional notice boards.

• It was spur of the moment when it started, by the way. Certainly not completely the case now, as project members are trying to keep in mind not to create any stub without asking first. -- AllyUnion (talk) 06:31, 16 Feb 2005 (UTC)

• "... for use on only a few articles": False: The stubs that are being created by the people in the Stub-sorting WikiProject are only being created when it can be shown that there are enough stubs to populate the stub category. Although things were a little too haphazard to begin with, especially before the creation of the Stub-sorting WikiProject, there is now a Wikipedia:Stub sorting policy, which says there should be a minimum of 100 articles.

  Baring in mind, that was Netoholic's idea. -- AllyUnion (talk) 06:24, 16 Feb 2005 (UTC)

  It use to be 10, by the way. -- AllyUnion (talk) 06:24, 16 Feb 2005 (UTC)

  And oh, that stub sorting policy has not been made official, and still in the draft stages. -- AllyUnion (talk) 06:31, 16 Feb 2005 (UTC)

• "many of these new "child" templates have come up for deletion": False. There have been a few "joke" templates at WP:TFD (such as Template:Cowstub and Template:Stub-sorting-stub), there have been a few accidental template creations that duplicated the scope of existing stub templates (such as Template:us-geo-stub and Template: Car-stub), and there have been a few that were created where the Stub-sorting WikiProject later decided that there were better ways to categorize things (such as Template: City-stub). Finally, there was also Template:Bush-stub, which was created by someone not involved with the Stub-sorting WikiProject. You've made it sound like there were dozens of stub templates that were inundating the WP:TFD process.

In my opinion, your descriptions went beyond just exaggerations in some cases. They were inaccurate descriptions of the stub template creation process, which it appears you have not investigated before you created your descriptions. BlankVerse ∅ 12:27, 13 Feb 2005 (UTC)

AllyUnion's responses are spot-on. Leave this section alone. Feel free to write Wikipedia:Meta-templates are teh best or somesuch. This page is to describe the negatives of using them, and no watering down of the scope creep section is welcome. -- Netoholic @ 04:26, 2005 Feb 18 (UTC)

I believe that meta-templates are considered harmful if and only if they are not used with the subst: included. Netoholic's example of the stub sorting is valid simply because it's one of the best examples he could find. I doubt that Netoholic would object if every stub template used {{subst:metastub}} instead of {{metastub}}. -- AllyUnion (talk) 22:17, 19 Feb 2005 (UTC)

If -all- that a template is used for is to "subst:", then I don't think the template should exist. In general it is much better to document the wikicode for the template "genre" on a project page for easy copy and pasting. -- Netoholic @ 03:29, 2005 Feb 20 (UTC)

Then we should delete {{vt}}, {{vb}}, {{vfd top}}, {{vfd bottom}}, {{vfd}} under your logic. -- AllyUnion (talk) 07:07, 4 Mar 2005 (UTC)

As the stub messages are being turned from readable semantically-marked-up code to less-readable stuff with DIVs flyign to and fro, I gotta as this.

As Jamesday pointed out, when a page's entry in the cache is empty, a database query is made to fetch that page. Then another one for each template it contains, and so forth. Why on earth aren't we caching the templates as well?

From what Jamesday said, this is the current process: Say page1 contains template, which contains meta-template. page2 also contains template. Then, if meta-template is changed, loading the two pages causes this to happen:

• page1(template(meta-template)) is dirty, needs to be refreshed. Fetch page1 from db.
  • Fetch template from db.
    • Fetch meta-template from db.
  • Assemble template(meta-template).
• Assemble page1(template(meta-template)).
• Serve (and cache) page1(template(meta-template)).
• Fetch page2 from db.
  • Fetch template from db.
    • Fetch meta-template from db.
  • Assemble template(meta-template).
• Assemble page2(template(meta-template)).
• Serve (and cache) page2(template(meta-template)).

This can't possibly be right, but it looks like this is what Jamesday is saying is happening. If so, no wonder meta-templates are such a performance hit. What's wrong with caching a version of the page with references to the templates, so they can be substituted into an already in-cache page without dirtying it? So we'd avoid dirtying zillions of pages and requiring them to be fetched all the way from the database again when, really, nothing's actually changed in them.

Am I way off base here? Someone set me straight. grendel|khan 02:07, 2005 Mar 4 (UTC)

1. It seems like we can change the way pages are cached instead of inconveniencing everyone by removing meta-templates. True?
2. If "glorified stylesheets" are converted into css classes, the classes have to be as visible and easy to use as the templates and also editable. - Omegatron 23:41, Mar 21, 2005 (UTC)

Stub templates aren't the only meta-templates in use. See Template:Message box. -- AllyUnion (talk) 07:08, 4 Mar 2005 (UTC)

No one's made any major changes to this in a while, nor refuted the basic conclusions. We already have some areas (like stubs) which are moving away from meta-template schemes now that the technical reasons have been described. One of the main reasons cited for other areas which are not yet adopting this principle is that it is not "policy". Well, let's start moving that way.

I'd like to move this from an "opinion" piece to a guideline. Namely, I propose renaming this to something like "Avoid using meta-templates", and replace the "proposed" line with one specifying this as an active guideline. I'd like to also link to it from the relevant template-related pages. -- Netoholic @ 22:31, 2005 Mar 21 (UTC)

The discussion has become moribund here not because there has been any consensus reached, but instead, it is because you have taken ownership of the article and not allowed any changes that you do not agree with. For example, your censorship of any mention of the use of "subst:". I do agree that Meta-templates should not be directly used, but without the use of such templates as the metastub template, you had the problem that existed before it was created where the newly created topic stubs had different wordings and formattings. Since the participants in the Stub-sorting Wikiproject have quit using the metastub template, that problem has returned.

I think that this is close to being ready to be Wikipedia:Semi-policy, but I think that it still needs some work and some rewording. Without those changes, I do not support changing its status. It certainly isn't ready to be official Wikipedia policy yet, which is what it sounds like you want to do. (Just a suggestion: I think that you need to become more familiar with Wikipedia:How to create policy, including "Do not rush" and "Consult widely - make a special effort to engage potential critics of the new policy".) BlankVerse ∅ 13:02, 22 Mar 2005 (UTC)

I am well aware of how to create policy, and don't see this as rushing. Please propose what you'd like to add. -- Netoholic @ 14:59, 2005 Mar 22 (UTC)