Former talk page archives

/Header /School /Newsletter /Awards Backlog drives Page Curation/Suggested improvement

This page has archives. Sections older than 45 days may be automatically archived by Lowercase sigmabot III when more than 5 sections are present.

So who's bright idea was it to ask brand-new NPPers, some still on trial periods, to review the work of people who have been doing it for years? Scanning down Wikipedia talk:New pages patrol/Backlog drives/October 2023, where "issues" are supposed to be reported, I already see a reviewer on a trial asking quibbling (wrongly) with a three-year veteran about MOS:DABNOLINK; and another patroller on a trial misidentifying the reviewer of an article. The logs show many more instances of unnecessary second-guessing that, contra the instructions, weren't followed up on that talk page.

Apart from being a bizarre way to use time when we're trying to reduce a backlog (surely we can all agree that enforcing the manual of style on disambiguation pages is not exactly a high priority?), it's a sure recipe for conflict. New NPPers on trial periods or in their first few months should be focusing on learning the ropes and getting the right permanently, not starting pointless arguments with other patrollers. Whoever is running this, please get some kind of guidance in place for who should re-review and when (if we even need to do it at all). – Joe (talk) 06:07, 10 October 2023 (UTC)[reply]

Its been done since at least the Nov 2021 drive. Obviously, it is meant for veteran NPPers checking the work of newbies. But, even some bad re-reviews (if swiftly corrected) will get the newcomer to understand some aspect of reviewing better, IMO. -MPGuy2824 (talk) 06:12, 10 October 2023 (UTC)[reply]

Looking through the talk pages of the backlog drives since then, I don't see a single valid concern with a review raised. New idea or not, I think my points above still stand, it's not a great one. – Joe (talk) 06:24, 10 October 2023 (UTC)[reply]

I agree with you on this point. What do you think we should do about it? Should we only exclude trial NPPs from the re-review, or should we also exclude new NPPs (those with 1 month of experience)? By the way, I'm also informing the drive's coordinators, @Hey man im josh, and @Illusion Flame, about this. 𝙳𝚛𝚎𝚊𝚖𝚁𝚒𝚖𝚖𝚎𝚛 ^{𝚍𝚒𝚜𝚌𝚞𝚜𝚜} 15:54, 10 October 2023 (UTC)[reply]

I never loved the idea of re-reviewing, but I got involved in this drive after it had already been partially set up. I'm open to just not doing it after this drive is concluded. Hey man im josh (talk) 16:05, 10 October 2023 (UTC)[reply]

My view does appear to be a minority, but I support the re-review system. The goal is to combat the possibility poor reviewing that a drive may cause. I believe we should have re-reviews only be for experienced reviewers. A message added to the top of this page could read: Re-reviews should only be completed by experienced reviewers very familiar with our policies and guidelines. New reviewers and reviewers on a trial period are discouraged from using the re-review system. - 🔥𝑰𝒍𝒍𝒖𝒔𝒊𝒐𝒏 𝑭𝒍𝒂𝒎𝒆 (𝒕𝒂𝒍𝒌)🔥 16:33, 10 October 2023 (UTC)[reply]

Speaking generally, the idea behind backlog drive re-reviews (for NPP, AFC, or any backlog drive that uses them) is to catch if someone is doing a bunch of poor reviews or reviewing too fast in order to get more points. Removing re-review requirements would get a different kind of complaint on this talk page, from those who are worried that backlog drives gamify things too much and lead to poor quality reviews. So take your pick of who you want to get complaints from :) –Novem Linguae (talk) 21:52, 10 October 2023 (UTC)[reply]

I agree with Novem Linguae and Illusion Flame. We could adopt a criteria in the next backlog drive to discourage brand new NPRs (<1 month of experience) and any still on trial from re-reviewing, but I don't think the system should be scrapped entirely because more often than not a second pair of eyes is beneficial (the aforementioned re-reviews are not great, but I don't think they form the majority and when I occasionally do cursory re-reviews I tend to find a couple that warrant AfD or a notability tag). Thanks. VickKiang (talk) 07:08, 13 October 2023 (UTC)[reply]

Probably best to just have a veteran spot check those newish reviewers with big numbers. For me a big part of what makes this job so painful is that it is impossible to do it perfectly and in a way that would be bulletproof to a magnifying glass applied later. Such could discourage reviewing. To do the full flow chart 100%, do a full wp:before on deletions, provide tags on all of the taggable problems, do everything that the fan clubs at AFD say was expected would take about 1/2 hr to 1 hour per article. If you look at the numbers, that would bring our backlog up to 100,000 within a few months. Sincerely, North8000 (talk) 14:49, 13 October 2023 (UTC)[reply]

Stepping back, I think we need to look into even more recruitment as all indicators point to an ever-growing backlog that we simply don't have the numbers to deal with. Backlog drives are all well and good, but we can't have people doing periods of intensive reviewing every four months as many will burn out after that. The dip after backlog drives is obviously desirable, but not if we have top reviewers taking breaks and the backlog ballooning up again. Although we were celebrating the last drive, we only cleared out around a third of the backlog, which has already replaced itself. A more sustainable approach may be more reviewers willing to review a larger number of pages consistently instead of a shorter period that could easily lead to burn out. All the best, Schminnte ^{[talk to me]} 22:51, 1 December 2023 (UTC)[reply]

We've tried a bunch of stuff to increase recruitment, and that has probably helped. For example, Wikipedia talk:New pages patrol/Coordination/Archive 7#NPP recruitment mass messaging idea. However it hasn't fully solved the problem, so I think mixing in backlog drives is important. @Raydann, are you still around? And is anyone else interested in helping with recruitment? The best way to recruit is probably to send out WP:MMSs to lists of active, experienced users we generate using quarry queries. Then that list needs to be filtered so that we don't double message anyone we've already messaged this year. –Novem Linguae (talk) 00:29, 2 December 2023 (UTC)[reply]

Thanks for the ping @Novem Linguae. Still here and still ready to help. I will look into making a list of eligible editors for the perm by quarry queries, and cross referencing them. The number of users whom I invited personally is not high, so it will not be difficult to exclude those editors. Furthermore, I will look for a way to exclude those who've already been invited by mass message. I suppose these are the lists from the previous outreach: New pages patrol/Coordination/Invite list, 2, 3, 4. Because I am not really familiar with quarry, I will try to understand how it works and maybe take the help of MPGuy2824. ❯❯❯ Raydann^(Talk) 18:34, 3 December 2023 (UTC)[reply]

@Raydann Hi! Whenever you’ve got the lists ready, ping me and I can send out an invite. Thanks! - 🔥𝑰𝒍𝒍𝒖𝒔𝒊𝒐𝒏 𝑭𝒍𝒂𝒎𝒆 (𝒕𝒂𝒍𝒌)🔥 18:46, 3 December 2023 (UTC)[reply]

@Illusion Flame - sounds good, I'll let you know when the list is ready. Also, @Novem Linguae, it looks like we discussed sending out New pages patrol/Coordination/Invite list 4 but never got around to it. And it also seems we never invited editors at New pages patrol/Coordination/Invite list 3. Therefore, I'm requesting WP:CSD#G6 on invite lists 3 & 4 so we don't get confused on who have been invited and who have not. ❯❯❯ Raydann^(Talk) 19:13, 3 December 2023 (UTC)[reply]

Sounds good. I fully support your efforts to clean things up. Thank you for taking the lead on it! Let me know if you need help generating fresh quarry queries. If we need bigger lists, we can always expand our criteria. Sounds like we didn't recruit enough folks last time to crush the backlog, and we also didn't get any spam complaints last time that I'm aware of, so I think we should consider greatly increasing the size of our lists. (And of course, we should be exceptionally careful not to double message anyone, else we will get spam complaints.) There are ways to programatically compare two giant lists and delete the overlapping names, so if you need help scripting this up, let me know. (It can be done quickly using array_diff in PHP, for example) –Novem Linguae (talk) 23:11, 3 December 2023 (UTC)[reply]

@Novem Linguae - I have prepared two quarry lists: Top AfD participants who aren't NPPs or Admins (1 month - Nov-Dec) & Top AfD participants who aren't NPPs or Admins (6 month) by forking another original quarry by MPGuy. The 1 month quarry list has 45 editors and the 6 month list has 316 editors. I definitely need help cross-referencing this list with other invited lists. Can you please guide me further? ❯❯❯ Raydann^(Talk) 02:06, 4 December 2023 (UTC)[reply]

@Raydann: I have a combined list of all invites and all the invites I've sent out. Just not available tonight to work on this. Hey man im josh (talk) 02:34, 4 December 2023 (UTC)[reply]

That is great. We can use the combined list for cross-referencing. But still take it easy, there's no haste! ❯❯❯ Raydann^(Talk) 02:47, 4 December 2023 (UTC)[reply]

Hey, you can use this tool to get items that are only in one of the lists. -MPGuy2824 (talk) 03:41, 4 December 2023 (UTC)[reply]

@Raydann. Thanks for starting this project! I combined the results of your two quarry queries and eliminated the duplicates. The result is here, although be sure to click "edit" so you can see the names properly formatted. Next step is to figure out what list(s) of "people we already sent to" to cross reference these two. @Hey man im josh, is your list comprehensive for this year (not missing anyone we sent to)? Think you can share it when you get a chance, and then I can do some more wizardry? –Novem Linguae (talk) 09:18, 8 December 2023 (UTC)[reply]

@Novem Linguae, @Raydann, @MPGuy2824: I created a page that includes my list in the first section (433 people), Novem's combined quarry queries (minus duplicates) in the second section (316 people), and a list of people that are only on Novem's list in the third section (255 people, 63 removed as overlapping).

The page can be found at Wikipedia:New pages patrol/Coordination/2023DecemberRecruitment (sorry, maybe could have had a better title). My list includes all of the NPR mass invite lists from this year were used to send out MMS invites, as well as my list of people that I personally invited, so as to avoid multiple invites to the same person. Hey man im josh (talk) 12:50, 8 December 2023 (UTC)[reply]

Note that I'm trimming a few names off the list as well (some blocked, some unsuitable, some with issues that disqualify them from eligiblity), as we've typically done in the past when creating these lists. Hey man im josh (talk) 15:12, 8 December 2023 (UTC)[reply]

I just removed 43 users from the list who had fewer than 3,000 edits. While I understand some of them could make great recruits, I think those are users who we should vet manually and more thoroughly. Hey man im josh (talk) 16:04, 8 December 2023 (UTC)[reply]

Awesome. So it sounds like the third section of Wikipedia:New pages patrol/Coordination/2023DecemberRecruitment is all filtered and ready for sending? –Novem Linguae (talk) 17:11, 9 December 2023 (UTC)[reply]

I'm not sure I want to say it's all filtered, that was just a quick lookover by me and a bare minimum of 3k edits as a cut off point. Do we want to discuss anymore criteria for inclusion/exclusion before doing so? I'm tempted to go through a part of this list myself and send out personal invites, as I do think individual invites work better than mass messages. Hey man im josh (talk) 13:37, 11 December 2023 (UTC)[reply]

Whatever works. As long as the plan doesn't get too complicated and we end up doing SOMETHING.

By the way, I've started a central page where we should log people we've contacted. That way each time we do this, we don't have to go "hmm, who have we already messaged? do you remember who has the list? do we need to combine it with any other lists?". Let's get in the habit of doing outreach, then immediately adding contacted folks to Wikipedia:New pages patrol/Coordination/List of users already invited. –Novem Linguae (talk) 15:01, 11 December 2023 (UTC)[reply]

@Novem Linguae I agree with @Hey man im josh. IMO personal invites are always better and more encouraging. Looking at the editor's talk page quickly gives us a glimpse of what kind of editor they are, and if they are suitable for reviewing or not. I'm happy to go over the list alongside josh, review the editor, and if all is well, leave an invite for them. So, am I clear to send out personal invites to eligible editors? ❯❯❯ Raydann^(Talk) 15:33, 11 December 2023 (UTC)[reply]

Definitely. Go ahead and begin, using any method you want.

By the way, I have some experience with email marketing, which I see as a lot like WP:MMS. Don't be too quick to write off the MMS strategy. The amount of labor per person is way lower than individual screening. At the end of the day, the more people we tell, the better. We need to get the message to a LOT of people if we want to find enough reviewers to get the backlog under control. –Novem Linguae (talk) 15:40, 11 December 2023 (UTC)[reply]

Agreed. I will start to add eligible editors to the MMS invite list, and in the meantime, personally invite solid candidates. Once the MMS list is ready, we'll send that too. ❯❯❯ Raydann^(Talk) 15:54, 11 December 2023 (UTC)[reply]

Maybe it would be healthier to have something that focuses on building more reviewers that are active on an ongoing basis. For example, longer term (over 1 year) there are only 7 reviewers that average at least 2 articles per day and only 19 that average at least one per day. Maybe add an database listing (and eventually awards) of who has gone the most months with reviewing at least 20 articles in each month. North8000 (talk) 19:02, 15 November 2023 (UTC)[reply]

Seems like a good idea to me. We can do this in addition to a backlog drive. Recognition coordinator @Dr vulpes, would you be interested in exploring this idea further (i.e. setting up a page somewhere, a quarry query) and then executing it (by announcing it and giving out barnstars)? –Novem Linguae (talk) 22:23, 15 November 2023 (UTC)[reply]

Just to emphasize I think that a visible updated listing is an important part of it. And maybe the 20 should be thirty, and maybe "30 day" periods would be easier to program than months. But I think that looking at ~1 month (or 2 or 3 month) periods is the right time frame. Nothing shorter than a month because even active folks might want to take a 2 or 3 week break or at least know that they can do that.North8000 (talk) 15:28, 16 November 2023 (UTC)[reply]

I noticed that Dr vulpes hasn't been the most active recently, so if need be, I can take over for any award distributions that need to be done. Hey man im josh (talk) 17:48, 16 November 2023 (UTC)[reply]

I think recognizing those who've done consistent reviewing over a period of time is a fantastic idea. I hope it's one that can be made to happen (realizing it's easy for me to say when I'm not doing the work). Best, Barkeep49 (talk) 17:52, 16 November 2023 (UTC)[reply]

@MPGuy2824, are you able to work your magic and whip up a quarry query for this? I really like this idea and, if nobody else is interested in implementing it, I'd like to do so. Hey man im josh (talk) 12:56, 8 December 2023 (UTC)[reply]

To solidify an idea/proposal it would be to: Add a database listing of those who have who has gone the most 30 day periods with reviewing at least 30 articles in each 30 day period. And later on add awards based on that. North8000 (talk) 15:31, 8 December 2023 (UTC)[reply]

Check this. Hopefully I have it right.

It is easier to do this on a monthly basis (instead of 30-day periods). Also, I've only counted for this year, and only upto November. Minor changes are needed to add the data for December (when the month is over). -MPGuy2824 (talk) 06:34, 9 December 2023 (UTC)[reply]

@MPGuy2824: Cool. I picked 30 days because I thought it was easier. But is that figure for number of months in the streak? if so, that first one says 53 years. Sincerely, North8000 (talk) 18:32, 9 December 2023 (UTC)[reply]

I replied to you by mistake (I meant to reply to Josh's message). The query that I linked to does not count the number of consecutive months that a particular reviewer has hit 30 reviews. It instead shows (for the period Jan 2023 - Nov 2023) the lowest monthly reviews for that reviewer. As you can see only 6 reviewers (ignoring the bot) reached 30 or more. -MPGuy2824 (talk) 03:23, 10 December 2023 (UTC)[reply]

@MPGuy2824: IMO getting the number of regular reviewers up would be be a big plus for keeping NPP on firm ground. This would mean folks who are watching and active and likely would "dial up" as needed when the backlog grows. What do you think about trying the "consecutive months that a particular reviewer has hit 30 reviews"? Sincerely, North8000 (talk) 16:38, 10 December 2023 (UTC)[reply]

The processing that you are asking for isn't easy to get via SQL (at least I don't know of an easy way to do it). It might be possible to do this via a spreadsheet program. You do need the raw data for that for which you can use the results of this query which gives you the reviews done by a reviewer in every month that they did a minimum of 30 reviews. Hope it helps. -MPGuy2824 (talk) 02:33, 11 December 2023 (UTC)[reply]

@MPGuy2824: Cool! Is there a way to take the result as a file? (spreadsheet or similar)? Sincerely North8000 (talk) 14:00, 11 December 2023 (UTC)[reply]

Absolutely. There is a blue "Download data" button, just above the results. There are many formats available to download, including CSV and Excel XLSX. -MPGuy2824 (talk) 04:51, 12 December 2023 (UTC)[reply]

What is says on the tin, the page seems to be duplicate of the PageTriage phabricator workboard and seems to be fairly infrequently updated. I think if we do want to keep track of the specific phabricator tasks associated with suggested improvements, we should use the {{tracked}} template in WP:PCSI (and maybe we can also encourage reporting directly to Phabricator via the the phabricator form?) Sohom (talk) 20:51, 3 December 2023 (UTC)[reply]

Looks like MPGuy2824 made the page. I'd be fine with marking it as historical (I didn't even have it on my watchlist, didn't know about it) but will defer to him. –Novem Linguae (talk) 06:43, 4 December 2023 (UTC)[reply]

The whole page was a section on WP:PCSI that I thought was cluttering up the page, which is why I separated it into its own sub-page. MB and Kudpung found it useful. As you can see from the history, I haven't been updating it much since both of them stopped being active.

It is linked from one of our newsletters and so I think we shouldn't delete it. Maybe convert it into a soft redirect to the phab workboard, with an additional link to the phab form that Sohom mentioned above. -MPGuy2824 (talk) 11:11, 4 December 2023 (UTC)[reply]

I've gone ahead and made it a soft redirect :) Sohom (talk) 01:18, 6 December 2023 (UTC)[reply]

Apologies for sounding like a broken record, but...surely there must be a way to automate NPP backlogs without depending on volunteers to carry the full load. A BOT could automatically reject unsourced articles and provide a relative response to the article creator, and the same for submissions with copyvios, poor sourcing, etc. BOTs can tag them quicker and more efficiently. Actually, a BOT could work hand-in-hand with the source rating BOTs such as User:Headbomb/unreliable.js. If we could automate the tedious parts and leave human reviewers to review only those articles that are notable, follow MOS, and are actually qualified for inclusion. AfC should only be releasing qualified articles, so maybe it is time to stop allowing just anyone to create articles in mainspace. Just a thought. Atsme 💬 📧 18:58, 4 December 2023 (UTC)[reply]

A bot that auto draftified articles with no citations might have trouble detecting if general references or a weird citation format are used. We do have a report for likely candidates though, to help reviewers spot them: Wikipedia:New pages patrol/Reports/Easy reviews#Probable "draftify because of no sources" candidates

A copyvio detection bot would suffer from the same problems with edge cases. Just because something is 50% on Earwig doesn't mean it's necessarily a copyright violation. It could be public domain, a big list of book titles, etc. We do use bots such as EranBot to look for likely copyright violations and put them in a report, but the final decision needs a human touch.

I don't think there's currently consensus to decline/reject/AFD articles simply because they have MOS problems such as no headings or a non-bolded title. MOS is not part of the flowcharts of NPP or AFC.

I like your thinking. We should always be looking for ways to make our processes more efficient. Too bad the problems are a bit on the complicated side. Many of the simple solutions have already been implemented, I think! –Novem Linguae (talk) 19:34, 4 December 2023 (UTC)[reply]

Every article in the new page feed is the work of a human being who, whether or not you like the result, almost certainly contributed it in good faith. Rejecting them without even having another human being look at them is a terrible idea and would be disastrous for the reproduction of our editor base.

Also your regular reminder that there is no policy prohibiting the creation of unsourced articles and draftifying articles solely because they lack citations is a questionable practice, at best. – Joe (talk) 15:29, 7 December 2023 (UTC)[reply]

The backlog is obviously surging again just a month after the previous backlog drive. I would argue that if feasible, a bot can be used to perform the Wikignome tasks (i.e., tagging for orphans, uncategorised, or detect basic cases where the title within body is not bolded, sort of like how AfD votes are detected, it won't work for complex cases but is better than nothing). However, bots shouldn't be used to automatically tag 50%+ copyvio articles per Earwig with revdel/copyright problem/G12 because of the significant rate of false positives (public domain, CC-BY-SA-3.0 compatible license, list of book titles, list of studies, and the like).

The same goes with drafting unreferenced articles- the automatic detections on PageCuration software doesn't work for articles with no footnotes but with general referencing or other unusual citation formats (which is a fairly high false positive rate). I also don't agree that every article that is unreferenced should automatically be processed by a bot. VickKiang (talk) 02:44, 12 December 2023 (UTC)[reply]

The wikignome stuff such as maintenance tags aren't a required part of the flowchart. It can be skipped to speed up reviewing. The important parts of the flowchart are CSD (which includes copyvio check), notability, and title check. –Novem Linguae (talk) 07:35, 12 December 2023 (UTC)[reply]

Yeah, I know, and often skip these processes myself. I am mainly arguing that is the only part that some (but not all) of NPRs do that I think can be (partially) automated, although I am am not sure 1) if it is feasible for a bot to constantly update data on whether a page is linked to a mainspace article and whether it has categories, 2) many editors do enjoy gnoming, and 3) it won't have a huge effect in reducing the backlog, only save the time of many diligent WikiGnomes.

By contrast, the key steps are often contentious clearly require human judgement (based upon current bot capabilities), and would have a much higher risk of having a damaging effect to newcomers due to false positives. For example, consistent false positive tags and flaggings of copyvio based only a bot would definitely deter newcomers and possibly dwindle the editing base. Thanks. VickKiang (talk) 07:47, 12 December 2023 (UTC)[reply]