Yobot (talk · contribs)

This is a proposal to revoke all approval for Yobot's bot jobs. Currently this bot is easily the most problematic on the whole project and problems date back to 2010 or earlier. The bot has been blocked 18 times previously, but each time the operator User:Magioladitis has managed to convince an administrator to unblock based on assurances that the bug had been fixed and will not occur again. The main issue with the bot is violation of the WP:COSMETICBOT rule, performing useless edits which fill up editor's watchlists.

When asked recently by User:SlimVirgin whether he could re-apply for permission to run these jobs, his uncompromising reply can be read here. So I am here to ask the community and BAG to revoke these permissions.

The advantage of forcing the operator to seek reauthorisation is that proper scrutiny can be given to each job and the method of working. It is possible that some of these processes are no longer supported by consensus. And it would give an opprtunity to spell out whether AWB's WP:GENFIXES may be used or not. I view this as a last chance for Yobot and Magioladitis's automated editing in general. If unresolved the next step is likely to be a community ban against all automated editing. — Martin (MSGJ · talk) 09:06, 20 December 2016 (UTC)

MSGJ Let's do this in steps then. I would like to check BRFAs 1-10 for starts. -- Magioladitis (talk) 09:09, 20 December 2016 (UTC)

And now I notice that some of the tasks requested for reapproval have already been abandoned by me. This makes no sense. Why should I get a new approval for tasks that were one-off or got outdated? -- Magioladitis (talk) 11:14, 20 December 2016 (UTC)

I think it would be fair to say that should this go though, you would not have to seek re-authorization for any task you didn't want to reopen... — xaosflux ^Talk 13:31, 20 December 2016 (UTC)

Xaosflux so it's going to be me the one to redo all the job because someone just does not feel right about it? -- Magioladitis (talk) 13:34, 20 December 2016 (UTC)

That started with a big if - if the community consensus is that these tasks should be deauthorized, and then you want to later restart them you would have to re-request. It will take more than "someone" to deauthorize all those tasks. — xaosflux ^Talk 17:58, 21 December 2016 (UTC)

MSGJ to explain that last part. -- Magioladitis (talk) 11:15, 20 December 2016 (UTC)

Yobot is the 4th (or 3rd) most active bot on English Wikipedia. So I don't this is the "most problematic on the whole project". "Citation" needed or as the shortcut fans would write "cn". I also recall some other bots that were completely blocked for malfunctioning. So I guess Yobot is in a better shape than them. Moreover, Yobot has earned 9 barnstars for its edits. -- Magioladitis (talk) 11:21, 20 December 2016 (UTC)

MSGJ per this section , I think Magioladitis raises a valid point. They state that bots should be dynamic, I couldn't agree more.

Consider vandalism removal, it's dynamic and as a result, is quite effective as a tool to keep wikipedia clean. Same could be said bot operations. If a bot is approved for say, fixing a reference in one article, and that reference happens quite a bit, why not add in an extra fix for, say another problematic reference in another article. The base mechanism of the bot already has approval, the second reference isn't problematic or controversial, so why not.

To be clearer, let me state it this way, if the change is non-controversial, nor problematic, why not let the bot operator of an approved bot add that change in to his or her bot .

Wikipedia certainly doesn't benefit by holding long week or longer discussions on such a change, just make the change ! KoshVorlon 14:34, 20 December 2016 (UTC)

Suggestion: Yobot's tasks, as far as I can tell, are valid, but if the bot finds that the error it is looking for is no longer present, it should skip the article. I believe that would fix all valid editor complaints. Mag: please don't fight this. You will lose, and the project will be worse for it.

One side note: Editors sometimes complain about valid, approved tasks that this bot and BG19bot perform, like removing spaces and non-printing characters for accessibility. These are approved bot tasks that are not cosmetic, even though they do not appear (to most people) to render the finished article any differently. – Jonesey95 (talk) 14:54, 20 December 2016 (UTC)

@KoshVorlon: The comparison with dynamic anti-vandalism bots is seductive, but ultimately it fails. No one is going to complain about reverting vandalism, but many editors complain about Yobot. I wouldn't have so much of a problem with dynamic working if the operator immediately stopped making an edit (I mean a "dynamic" one without community consensus) that someone complained about. The fact that someone is complaining means that it wasn't actually all that uncontroversial and consensus should be sought. That is generally not the response from Yobot's operator. More often than not there is a refusal to stop and a claim that the task has consensus when really it doesn't, it was just something that happens to be in genfixes, and the contents of genfixes are not reviewed by BAG. And there is the problem, some Yobot tasks rely on genfixes to work, but genfixes, in general, doesn't have community consensus.

The issue that led to Yobot's current block for making cosmetic-only edits was substituting template redirects, which most editors consider cosmetic edits. Now I understand why AWB needs to maintain a list of template equivalences - some processes depend on that, but I don't understand why bots need to be running around actually substituting them. I don't even understand why AWB human operators are being suggested to do this, let alone bots. So in short, I think it right that at least current Yobot tasks implementing genfixes should be re-examined. SpinningSpark 15:50, 20 December 2016 (UTC)

Spinningspark most of the complaints I get are not from the "usual suspects" but for adding "orphan" in pages. This is because many people do not understand the concept of "orphan". Similar,I get many complaints who do not exactly disagree with me or my bot but with the policy/guideline itself. As I wrote WP:GENFIXES links to all templates affected. The documentation of each templates explains what it does. You wrote that genfixes do not have enough documentation. This is not true. There are many links. Moreover, this page was created after AWB's code expanded and not the other way around. It was difficult sometimes to even determine what each piece of AWB's code does. -- Magioladitis (talk) 16:01, 20 December 2016 (UTC)

I don't know where you think I said that genfixes doesn't have enough documentation, but if it's where I said that it does not have an audit trail then that is a very different thing. SpinningSpark 16:15, 20 December 2016 (UTC)

SpinningsparkA large part of the documentation was written by GoingBatty who is not of the developers. It's very difficult to keep the documentation, the changelog and the program all up-to-date. I also keep a log of all changes that affect CHECKWIKI. See how many cases are now fixed compared to 2014. Yobot is by far making much fewer null edits than in the past. -- Magioladitis (talk) 16:17, 20 December 2016 (UTC)

I think it's clear we need more copycats of Yobot and more bot operators involved. -- Magioladitis (talk) 16:34, 20 December 2016 (UTC)

I'm in support of MSGJ's request here. It has become something of a mess; let's clear the slate, and get some decent scrutiny (and community consensus) for each of the proposed tasks going forwards. Hchc2009 (talk) 17:02, 20 December 2016 (UTC)

Hchc2009 which task is not clear to you then? -- Magioladitis (talk) 17:05, 20 December 2016 (UTC)

• Support revoking approval for Yobot's jobs. Looking through some of the talk-page archives (by no means all), I've found around 130 threads about Yobot's or Magioladitis's cosmetic-only edits or ref reordering, and this does not include complaints about other issues. Magioladitis is deaf to all concerns, blaming issues on a bug that has been fixed or is being fixed. Other bots do not attract this number of complaints.
  The discussions have wasted hundreds of hours of volunteers' time; the most recent AN is at Wikipedia:Administrators' noticeboard/Archive279#Magioladitis, 28 January – 19 February 2016. Yobot has been blocked 18 times because of it and Magioladitis five times. Better to wipe the slate clean and start again than to argue over the details. SarahSV ^(talk) 18:01, 20 December 2016 (UTC)

SarahSV There is no consensus against ref reordering yet. I am now reviewing the discussion. -- Magioladitis (talk) 18:18, 20 December 2016 (UTC)

• Support, if that was not clear enough from my previous comments. SpinningSpark 19:14, 20 December 2016 (UTC)
• Oppose this seems to be the continuation of disagreements between SV and Magioladitis by other means. If anyone has any issues, let them challenge the individual BRFA. A blanket revocation, causing Magioladitis (and others) hundreds of hours of work to re-do them all seems a sledgehammer to crack a nut. All the best: Rich Farmbrough, 19:37, 20 December 2016 (UTC).
  

Well the first problem there is that of the last 100 edits Yobot made prior to being blocked, not one of them indicated which approved task the bot was implementing. So how is an ordinary editor supposed to know which BRFA they wish to challenge? Or even find out that there is such a thing as BRFAs in the first place? SpinningSpark 14:27, 22 December 2016 (UTC)

For most bots, the use of a task list on their userpage and good edit summaries normally suffice. When there are many tasks it can get confusing. It is fairly easy to incorporate this to an edit summary (example) - would having that type of information help? — xaosflux ^Talk 15:05, 22 December 2016 (UTC)

@Spinningspark and Xaosflux: I replied at User_talk:Spinningspark#Yobot.27s_tasks. -- Magioladitis (talk) 15:50, 22 December 2016 (UTC)

• Oppose per Rich. I'm all for additional scrutiny given the recent disagreements, but it should be on the proposer to specify what tasks are problematic or may no longer have consensus. Especially for such an active bot, the solution to disagreements can't be to just revoke it and start from scratch. Besides that, I think Jonesey95's point about how removing white space for accessibility reasons hasn't been given enough attention. Just because it doesn't change the output of your page doesn't mean it's cosmetic for other editors using screen readers and other devices. Wugapodes [t^hɔk] [ˈkan.ˌʧɻɪbz] 20:57, 20 December 2016 (UTC)

  Your statement makes no sense. If the change does not change the rendered output, that's the very definition of WP:COSMETICBOT. Screen readers would still interpret the output the same way. HTML syntax allows rendered out put of up to one space using whitespaces. The rest need to be coded in with HTML syntax. If you observe, the spacing between the start of this sentence and the previous one, is more than one in the edit window, but still renders as one in the output.—^cyberpower_{Merry Christmas:Unknown} 03:07, 21 December 2016 (UTC)

I suppose I should have linked directly to the BRFA for BG19bot 9, where white space removal by a bot was approved to fix an accessibility problem.

That said, there are many complaints about Yobot making cosmetic edits because it fails to find something to fix. Here's one on the bot's talk page right now, from just nine days ago. As far as I can tell, Yobot made a purely cosmetic edit in that case, and Mag replied "The bot failed to fix this (link). Changes to CHECKWIKI affected bot activity. I am on it." If the bot fails to find something to fix, it should take no action. I am not a bot admin, but this should not be hard to change.

Can someone here please help Mag fix Yobot so that it skips edits instead of making purely cosmetic edits? Other bots appear to do it just fine. For an example of an editor who has no problem with this issue, see the query at User talk:GoingBatty/Archive5 about "Civilian casualty ratio" and the response from the bot operator. Maybe Yobot is different somehow? – Jonesey95 (talk) 06:54, 21 December 2016 (UTC)

My plate is full. IABot generally doesn't reformat a cite template unless it's actually doing something with it.—^cyberpower_{Merry Christmas:Unknown} 14:25, 22 December 2016 (UTC)

• Comment I don't recall a specific BRFA being the cause to when people start complaining about his cosmetic edits. If Yobot has to go thru BRFA's again, will Martin, SlimVirgin, Spinningspark and Carl be fighting tooth and nail or will the BAGs be able to do their job? Bgwhite (talk) 02:44, 21 December 2016 (UTC)

• As far as I'm concerned, only currently live tasks that have approval to apply genfixes need re-evaluating. In fact, I would be happy if just Yobot's use of genfixes was discussed in general rather than each individual task. Everything else can be nodded through. I can't speak for others of course. SpinningSpark 03:19, 21 December 2016 (UTC)

  @Spinningspark: I think you would find that most if not all of Yobot's jobs involve genfixes. So you would effectively be reviewing all the jobs, which is why I believe a clean start is necessary. — Martin (MSGJ · talk) 09:35, 21 December 2016 (UTC)

  That may be so, and it might be helpful if Magio declared which jobs are applying genfixes if that is different from the ones on which he has explicit approval to apply genfixes, but I think it is going to be a lot more productive to have an overarching approval (or denial of approval) on the use of genfixes, rather than have Magio repeat the same rationale over and over in multiple applications. From his comments, that is clearly going to be a great source of unnecesaary annoyance to him. SpinningSpark 11:45, 21 December 2016 (UTC)

• Comment I've seen this get out of hand at ANI, recalling User:Bgwhite and User:Magioladitis quibbling over User:Yobot. To be honest I found it to be a bit childish, and both sides could have handled it better. I'm not going to support or oppose this because a bot op should be able to comply with basic bot policy. It's not difficult. If COSMETICBOT is the problem, use the API's parser to compare rendered output between the new and old version. If they are the same skip the edit. The API has a beautiful parsing function bots can use. It certainly makes IABot's life easier when rendering embedded templates in cite templates.—^cyberpower_{Merry Christmas:Unknown} 03:15, 21 December 2016 (UTC)

C678. True. But I think the problem is that some people would oppose changes that actually change the html output. -- Magioladitis (talk) 08:53, 21 December 2016 (UTC)

Well in that case I would have to say a widely advertised RfC is in order to see what the community really wants. I wasn't happy when I encountered Yobot initially as it kept moving a comma around in the article I created.—^cyberpower_{Merry Christmas:Unknown} 14:25, 22 December 2016 (UTC)

Support. Yobot has been operated in a problematic way for years; the operator has refused to fix several longstanding problems. The first is the continual pattern of cosmetic edits. The second is lack of clear edit summaries. BAG missed an opportunity to remedy/prevent these issues in the first round of bot approvals for Yobot. Usually these issues aren't a problem because other operators follow best practices. A second round of approvals could remedy the situation now that the pattern of bad edits is known. — Carl (CBM · talk) 13:50, 22 December 2016 (UTC)

• Support. Extremely problematic behavior from this bot for years. I've run into downright head-scratchers here regularly, and some of these approvals are incredibly broad to a degree that would never be approved today. It's clear this bot operator needs some oversight, and all these blanket approvals aren't sufficient in providing that. I think this is the best course of action for both the community and the bot operator, as it will give them a better idea of how they must operate to remain inside the bot policy if BAG members are overseeing their tasks more regularly. ~ Rob₁₃^Talk 06:46, 27 December 2016 (UTC)
  • @Magioladitis: How willing would you be to take a look through all bot tasks and voluntarily revoke Task 16 and others that are blanket approvals (which have been problematic in the past from Yobot)? I could be persuaded to oppose a blanket revocation if good-faith steps were made to get rid of the problematic approvals. If this is something you'd be willing to seriously consider, I'll try to put together a list of what I think needs re-examination. It does go beyond task 16, but possibly not that far beyond. Some of them are really ancient blanket approvals that would be unlikely to be approved in modern times. ~ Rob₁₃^Talk 09:41, 30 December 2016 (UTC)
    • BU Rob13 examples of blanket approvals? -- Magioladitis (talk) 09:43, 30 December 2016 (UTC)
      • @Magioladitis: Task 9 approves project tagging for all projects, including any method of changing parameters, with no restriction on how complicated or cosmetic that may be. Compare, for instance, to my many project tagging bot approvals in the modern day. These days, such tasks are typically sent through one-at-a-time to check consensus, at the very least. I haven't checked systematically yet, but that one springs to mind. ~ Rob₁₃^Talk 09:47, 30 December 2016 (UTC)
        • BU Rob13 If I recall correctly this was asked from you because you were not 100% familiar with the Wikiproject concept by that time and in a perspective at some point to grant you a blank approval after the Wikiprojects get aware of your bot and you become more experienced. But anyway, this one is from the tasks I asked people to take over. You are of on my candidates to take this. -- Magioladitis (talk) 09:57, 30 December 2016 (UTC)
          • Actually, I never requested blanket approval for this sort of task because I just don't think it's appropriate. There are so many permutations of how WikiProject tagging work that I see it as worthwhile to have a BAG member check over the consensus, determine if it's really similar to past tasks in implementation, and then either speedy approve or request a trial based on how things look. I will freely admit I specifically have an issue with you doing certain tasks (mainly, Task 16), because of past history with cosmetic changes. But this is something I'd want revoked from anyone, because I don't think it's an approval that stands up to community norms of bot approvals in the modern day. I'd be more likely to oppose a blanket revocation if you voluntarily gave up the tasks that I think are problematic regardless of who the botop is. (And you're welcome to send projects that need tagging done my way, although I'll accept/reject such requests based on my own standards of what's appropriate for project tagging. I learned long ago not to take projects at their word on tagging.) ~ Rob₁₃^Talk 10:05, 30 December 2016 (UTC)
          • BU Rob 13 As I wrote: Yes, the biggest problem is Task 16 and I am OK if Yobot stops doing most of it. I have skip conditions for CHECKWIKI #16, #78 and some others which Yobot can continue doing. I am OK if BG19bot, MenoBot and other do the other tasks instead of Yobot. Some other tasks I already have given or tried to give to Dexbot. I am usually the bad guy because I always jump into the new errors. Notice, that the entire logic for fixing new errors is based on Yobot's bugs that were reported and fixed. -- Magioladitis (talk) 10:22, 30 December 2016 (UTC)

• Comment I think the only problem is Task 16 due to its nature. I can work with Bgwhite on which CHECKWIKI errors should be discontinued in bot mode after recent changes. -- Magioladitis (talk) 10:42, 27 December 2016 (UTC)

The issue of cosmetic and unapproved edits has come up over and over with Yobot, for numerous bot tasks, not just one CHECKWIKI error. In my mind, the best solution would be to disable all "general fixes" and only apply the very specific change described in each bot request. If a bot request is not specific enough to say the precise change that will be made, it can always be resubmitted. — Carl (CBM · talk) 17:41, 30 December 2016 (UTC)

• Above, I was hoping to go through the tasks and work out a way to only support rescinding some of the tasks, but I just saw something that convinced me full revocation is the way to go. The bot operator has now stated in Special:Diff/757725689 that he routinely runs Yobot after making substantial untested changes to how it operates. That alone should be enough to rescind all approvals. No wonder there have been such large error rates! The bot approvals process can provide the structure to test changes, and with more narrow approvals, I think Yobot can get to the point where it runs smoothly. ~ Rob₁₃^Talk 11:37, 1 January 2017 (UTC)
  • In the last year there were 4 or 5 changes that affected the lists. I should be more careful to see that some lists exploded. I am aware of that that's why I asked that the lists now contain the number of items in their edit summary. I feel like you Wikipedia:Wikilawyering. -- Magioladitis (talk) 11:55, 1 January 2017 (UTC)

• Comment. Pardon me if this seems like a "Captain Obvious" thing to point out, but as someone who has used AWB a lot for the past 2 1/2 years, I know a little about the tool.  :) This particular factoid never seems to make an appearance with respect to Yobot. That factoid is: "Skip only cosmetic changes" is a feature in AWB, and it's a mere checkbox. Yobot uses AWB. So, why doesn't Yobot have this checkbox checked? And if it does have it checked, why is AWB failing to skip cosmetic edits? Stevie is the man! ^{Talk • Work} 18:51, 4 January 2017 (UTC)
  • Stevietheman I have filled a bug report in April this year T132286 -- Magioladitis (talk) 19:05, 4 January 2017 (UTC)
    • On this aspect, I truly feel your pain.  :) Stevie is the man! ^{Talk • Work} 20:04, 4 January 2017 (UTC)

Take note that I never denied help and never claimed that the underlying huge problem of not having correct skip conditions has been solved. This can be resolved better in an eternal way similar to what WPCleaner does: It sends a call to the server and checks whether the error is still reported. This is time consuming ofcourse. For instance, right now the time to check whether the error has been fixed it takes me more time than to run general fixes in the page. NicoV has done great effort to reduce the time. We even found errors in the code last year. We are actually making effort on this by exchanging emails daily. This is not only the job seen on-wiki. Some people may not believe it butBgwhite, NicoV and me spend several hours per day to fix things and synchronise three different programs. It's not an excuse when I say that I am not sure I have more free time to give. My entire is this.

I even tried to write which AWB function is called to fix each error and wrote it down at User:Magioladitis/AWB and CHECKWIKI. In some cases more than one function is needed or in other case the function called does much more thinhs than just fixing the error. -- Magioladitis (talk) 19:16, 4 January 2017 (UTC)

Magioladitis, the issue that puzzles people is that other bots don't have this trouble. Is there something different about Yobot that causes the problem? SarahSV ^(talk) 19:29, 4 January 2017 (UTC)

SarahSV what I think Bgwhite does is to do almost every single error semi-manually except a very small number done by his bot. It's amazing but also very cruel. Bgwhite spends hours doing everything manually. If we had more volunteers participating in the CHECKWIKI project we may not needed the bots at all. But 2 people fixing 1,000 pages daily (including weekends) it's cruel. I think he can speak for himself ofcourse. Other, like Meno25 tried to help but for various reason the are not so often on-wiki anymore. On the other hand, I invested my time reporting errors and trying to find patterns to fix as much as possible by bot. It's true than in some cases I reached the limits of AWB i.e. there were cases not fixed. I don't claim to to be an expert programmer. I usually give tasks to others. -- Magioladitis (talk) 19:48, 4 January 2017 (UTC)

Magioladitis, as I understood our conversation here and here, the cosmetic edits are a main feature for you. You see them as important because you want the wikitext to be perfect, so you don't switch them off. Other editors do switch them off. They check "Skip only cosmetic changes". Is that not the case? SarahSV ^(talk) 19:56, 4 January 2017 (UTC)

SarahSV There is no way to switch them off. There is a way to skip a page if only "cosmetic changes" happen. But this definition in AWB does not include bypassing redirects. -- Magioladitis (talk) 20:02, 4 January 2017 (UTC)

SarahSV Just a clarification: Yes, I enjoy doing little edits but this does not mean I do them with my bot account. By bot has a very specific approval and I try to respect that. There is not fun in sitting and seeing a program do the minor edits for me :) Moreover, that I enjoy this that does not mean that this is the main thing I do or that I do it that massively as it is implied by some others. Most probably, I combine my editing with something useful. -- Magioladitis (talk) 20:16, 4 January 2017 (UTC)

@Magioladitis: Okay, so I return to the question. Yobot makes lots of cosmetic-only edits. Other bots don't. AWB allows you to skip cosmetic-only edits. Why doesn't Yobot skip them, as other bots do? SarahSV ^(talk) 20:25, 4 January 2017 (UTC)

SarahSV I do. With the existing AWB code bypassing a redirect is not cosmetic. It changes the html output. -- Magioladitis (talk) 20:29, 4 January 2017 (UTC)

@Magioladitis: I'm not talking only about bypassing redirects, but about "cosmetic edits only" in general; for example doing nothing but add white space. Yobot does that a lot, so clearly the bot is not skipping pages when there are "cosmetic changes only". SarahSV ^(talk) 20:37, 4 January 2017 (UTC)

@SlimVirgin: there's a setting for that too: "Skip Only whitespace" -- this can be checked at the same time. Magio, do you check this for Yobot? If not, is there a reason why not? Stevie is the man! ^{Talk • Work} 20:41, 4 January 2017 (UTC)

Of course, as a bot operator you are in complete control of the code that you run. There are several straightforward fixes to the problem, including (1) comment out all tasks other than the one the bot is meant to perform, or (2) make a test directly after the task that was supposed to be performed, and skip the page if that particular task did not change the page source. It is not reasonable for a bot operator to continue running flawed/buggy bot code, while simultaneously claiming that they are not able to fix that code. That is the root of the issue here: when an error is found that is easy to duplicate, the bot should not be run again until the code is fixed. — Carl (CBM · talk) 20:14, 4 January 2017 (UTC)

CBM True. As I wrote above I am ready to have my list of CHECKWIKI errors fixed by bot drastically reduced. For example User_talk:Magioladitis#Errors_7.2C_19.2C_25.2C_83. -- Magioladitis (talk) 20:19, 4 January 2017 (UTC)

That does not solve the problem, which is that the bot will still save edits when it should not, unless the code of the bot is changed to prevent it. The template redirects issue that you mentioned dates back to 2010, but the bot continues to make the same error: [4]. There has been plenty of time for you to fix the problem; is time for all the BRFA approvals to be rescinded, and re-approved only with clear guarantees that this problem will not re-occur. — Carl (CBM · talk) 20:24, 4 January 2017 (UTC)

@CBM: Given Yobot uses AWB, that is technically "somebody else's code" (to a large extent save the specific fixes). Also, it already has a checkbox to skip cosmetic edits, except that it won't treat template redirect bypasses as cosmetic, unfortunately. If that bug in AWB was corrected, and Yobot given a test period to run after that, maybe BAG could reasonably decide to not throw the baby out.... Stevie is the man! ^{Talk • Work} 20:34, 4 January 2017 (UTC)

No, because bypassing redirects are not the only "cosmetic edits only" edits that Yobot makes. Why is this so hard to get across? SarahSV ^(talk) 20:41, 4 January 2017 (UTC)

It has gotten across to me. See my reply above about whitespace, noting that a whitespace only change will often be considered non-cosmetic by AWB, so skipping both cosmetic only and whitespace only seems the best way to go. Stevie is the man! ^{Talk • Work} 20:44, 4 January 2017 (UTC)

AWB might be somebody else's code, but once a bot operator chooses to use it they take responsibility not to use it in ways that cause problems, and to fix problems that arise. In this case, Magioladitis has refused to fix the issues with the bot for several years. If AWB is not an appropriate tool for Yobot, he should use a different tool. Other AWB operators manage to avoid the issue, by and large - I see many AWB bots on my watchlist, but not making the same errors as Yobot. All in all, this is why I think that the focus should be on Yobot, rather than AWB. — Carl (CBM · talk) 20:49, 4 January 2017 (UTC)

@Stevietheman: It seems to be hard to get across that a "cosmetic only edit" is one that doesn't change the appearance of the page to a reader. Anything falling under that definition must (a) be done at the same time as a substantive edit; or (b) the bot needs special approval.

This discussion misses the point that M seems to want to make these cosmetic edits, because he wants the wikitext to be perfect (in his view). That's why he often adds a non-breaking space, so that he can also fix a redirect or add white space. If he didn't want to do this, it would not have been happening for eight years. It's that underlying issue that needs to be addressed, especially given that he says it is taking up all his time. We have a lose-lose situation here, and talk of bugs and code is missing that important aspect. SarahSV ^(talk) 20:52, 4 January 2017 (UTC)

I know what a "cosmetic only" edit means to the wiki community, but "cosmetic only" in AWB merely means if the HTML hasn't changed. That's why normally with using AWB, I also skip whitespace only and oftentimes skip only minor genfixes so it makes my job easier (but not as easy as proceeding with my eyes closed) to pinpoint a total save that doesn't have at least one substantive change in it. That's why it's hard to use AWB for unattended bot purposes. I won't speak to what Magio wants. I'm trying to see if a better technical approach, combined with bug fixes, would resolve this. Stevie is the man! ^{Talk • Work} 21:05, 4 January 2017 (UTC)

Some whitespace fixing maybe coming from main functions while trying to do other stuff. Skip if only whitespace has prevented legit in the past. For example error #89 is about adding whitespace. -- Magioladitis (talk) 20:54, 4 January 2017 (UTC)

If Yobot is being used to fix an approved whitespace-oriented error, then, obviously, don't skip whitespace only on those runs. Otherwise, it seems best warranted to skip whitespace only and maybe lose out on a few legit edits to get gains in the bot's observed integrity. Stevie is the man! ^{Talk • Work} 21:31, 4 January 2017 (UTC)

SarahSV I don not want my bot to do "trivial edits". You miss that point. -- Magioladitis (talk) 21:02, 4 January 2017 (UTC)

@Magioladitis: I began collecting diffs of threads some time ago, in anticipation that this would go to AN/I or ArbCom at some point. I intended to post them on a user subpage in case they were helpful. I ended up finding so many that I didn't post the page, because it was a wall of shame. For eight years, editors have been stuck in a loop when dealing with you. "Don't do X." Sorry, it's a bug. "Don't do X." Please define X. "Don't do X." Sorry, it's a bug.

I would like to identify the underlying cause of that miscommunication, so that we can escape the loop. SarahSV ^(talk) 22:23, 4 January 2017 (UTC)

SarahSV Well, for eight year I am helping in improving AWB. I am a volunteer. This is the best I can do. I am not even a programmer. I learned to program some stuff in order to help and stop bugging the only 2-3 programmers who work with AWB. AWB is open source and I never denied patches. Thanks for thinking that I am super-human (judging by the time I spend online?) but I am afraid I am not. Many other bot owners just stopped and bot task were abandoned. I am still trying to find my successor. Yobot is not the best we have. It's still more optimal from many other things around. Do you know someone that could write me a custom module that will load all CHECKWIKI errors and have skip checks for them? I would be more than happy to run this module. I am writing you that my make effort to make things better, I provide you proof. You obviously want more. :) PS In latest Hackathon I organised in Greece I even asked students to help out with some of the bugs. -- Magioladitis (talk) 23:10, 4 January 2017 (UTC)

At the risk of being overly charitable, those bugs maybe really were bugs. As new fix tasks come aboard, new bugs become likely, and they have to be addressed each time. As for the definition misunderstandings, I may have trouble being as charitable, because I think it's hard to escape understanding of what a substantive edit is and isn't in most cases. Stevie is the man! ^{Talk • Work} 23:36, 4 January 2017 (UTC)

@Stevietheman: Which AWB fix/tweak would resolve the following editing error: [5]. The bad edit summary compounds the problem, since there is no way to tell what the bot was trying to do, but certainly that edit was not what the bot was supposed to do. There was no accompanying whitespace change, I think. — Carl (CBM · talk) 22:53, 4 January 2017 (UTC)

I don't know how the summary is being set, whether manually edited, or using Find/Replace or module, so I can't speak to that, but for this to be saved by the bot at all, given "Skip Only cosmetic changes" is checked, is due to an as-yet unfixed AWB bug. I suspect that was part of a run where the intended fix wasn't necessary for that article by the time the bot got to it, but the template redirect bypasses were left -- and they sailed through because there's generally no reasonable way for AWB to stop them -- unless and until this bug is fixed or if genfixes aren't done at all for the time-being (but then, other potential substantive fixes won't happen). Stevie is the man! ^{Talk • Work} 23:09, 4 January 2017 (UTC)

I'll limit my comments to the technical issues only. Current approach for these kinds of bot tasks aiming to fix a specific error seems to be:

1. get list of articles having the error from a reporting tool (e.g. CHECKWIKI), process all the articles
2. use a fixing tool (e.g. AWB) to process article by applying the tool's standard fixing routines
3. hope that the tool found and fixed the error
4. apply some basic or general checks to skip the edit in certain situations

So the fundamental technical problem as I see it is that an accurate bot task with a very low cosmetic edit rate needs to have two task-specific checking steps:

1. get list of articles having the error from a reporting tool (e.g. CHECKWIKI), process all the articles
2. check page currently has specific error, skip if it doesn't
3. use a fixing tool (e.g. AWB) to process article by applying the tool's standard fixing routines
4. check page no longer has specific error, skip if it still does

CBM's suggestion of only running selected genfix functions has some merits, however there are some functions that can make both significant and sometimes trivial/cosmetic changes, so again without specific checks, while that approach might somewhat reduce cosmetic/trivial error rate, it doesn't fully solve it. Other discussion about template redirects etc. to me misses the fundamental point about specific checks; one source of cosmetic/trivial edits could be template redirects today, but tomorrow could be stub whitespace, excess template pipes or one of potentially dozens of other items.

So what I'd expect is that the bot operators are asking other editors/contributors to the tools and editors with technical skills for help to configure these specific checks. Magioladitis' last comment suggests he's willing to go down that route. Rjwilmsi 23:29, 4 January 2017 (UTC)

@Rjwilmsi: I agree this could be looked at with a wider lens, and should be, but the specific case of template redirect bypassing is a pure cosmetic edit in the AWB sense -- no difference in HTML output. Fixing that won't fix everything that is possible to be fixed for the bot to run perfectly or nearly so, but nevertheless, fixing this will be a welcome and true improvement for AWB users in general, and somewhat ameliorates Yobot's troubles. Stevie is the man! ^{Talk • Work} 23:52, 4 January 2017 (UTC)

@Rjwilmsi: A standard algorithm to avoid this is to test the source of the page before and after the desired edit of the bot is applied. If there is no change in the source, then the page should be skipped. If there is a change, then general fixes can be applied. This algorithm is entirely local within AWB and does not require double-checking whether a page is or should be on any external list. It only requires saving/rendering a copy of the soruce code before and after a particular fix is run, and then comparing the two versions. So it is also very efficient. Is there a technical reason this cannot be implemented? It seems like it would resolve most of the cosmetic edits easily. — Carl (CBM · talk) 01:01, 5 January 2017 (UTC)

• This discussion should probably be postponed until the resolution of the Arbcom case because it's unlikely that Yobot will be unblocked before then, if at all. Ramaksoud2000 ^{(Talk to me)} 05:23, 5 January 2017 (UTC)

Well, we're discussing some technical principles so I think there's value in discussing even if any unblock wouldn't be soon.

@Stevietheman: The phab AWB ticket T132286 is now fixed, the "only cosmetic changes" skip check is now working e.g. for template redirects.

@CBM: In summary I'm proposing task-specific skip checks, you propose task-specific logic to fix the targeted error. They would get to the same place, but with AWB one is easier than the other, which I'll explain detail below. So I don't disagree with you, am trying to describe an easier/simpler means of getting there.

To be clear, I think you mean checking the rendered page to see if there is a visible change. Yes, AWB already has this, though only to check the entire AWB edit, not a first part of it as you suggest. (It requires two API calls so technically I would not call it particularly efficient, but nobody here counts API calls so that's no matter) Secondly your suggested approach means the bot task has to develop its own code/logic to make the fix, separately for each task, test that for visible change, then apply genfixes. This would be possible in AWB but would require coding to have a custom module/external process to run a fixing routine developed per bot task, then apply the cosmetic change check, then only apply genfixes. So genfixes couldn't be used to attempt the fix of the error, which realistically means the fix logic would be trying to reimplement portions of genfixes.

That approach is unlikely to be something that bot operators such as Magioladitis have the technical knowledge to do. I am thinking the most feasible in AWB, and still fundamental technical solution, is for the bot task to add task-specific error checks. If the error comes from CHECKWIKI then it ought to be possible to import the error checking logic, which is probably based on regular expressions, into AWB, which if just regular expressions would be a matter of configuration of AWB rather than complex code. Run the check before to ensure the error is there, run again after genfixes to confirm error has gone away. Rjwilmsi 07:21, 5 January 2017 (UTC)

@Rjwilmsi: Checking the rendered page via the API would work, as well, but of course it is much slower. I was talking about something simpler, just checking the source of the page. As I understand it, the problem with Yobot is that it wants to apply a particular change to an article, and then apply general fixes. But it has no error checking to tell whether the initial change actually changed anything, so it just goes ahead and makes the general fixes and saves the page anyway. This is why it matters to keep the lists very fresh. At least, this is how Magioladitis has explained the problem several times. If the system could tell whether the original, desired change actually changed the page's source code, then the system would be able to tell whether to save the page or skip to the next one. (The original change might be a specifically approved cosmetic change, as well.) Of course, importing the CHECKWIKI logic would also work. — Carl (CBM · talk) 11:17, 5 January 2017 (UTC)

Carl, "the problem with Yobot is that it wants to apply a particular change to an article, and then apply general fixes" I think the problem is that "it wants to apply a particular change to an article" by applying general fixes, so then has difficulty determining if the change(s) made included fixing the issue or not. Bearing in mind that a visible change could still be cosmetic, and a visible and/or useful change may still not relate to the intended error, so there is no single generic check that can determine that; we agree that by some means a specific check as to whether the desired fix happened is needed. Rjwilmsi 11:44, 5 January 2017 (UTC)

@Stevietheman and CBM: FYI, Rjwilmsi fixed the "Bypasing redirects should be a cosmetic change". This means edits like the one Carl provided should not happen again. -- Magioladitis (talk) 09:46, 5 January 2017 (UTC)

Yes, with a working "skip if change doesn't affect the rendered page" check, you can be sure that the edit changed the page appearance, so will avoid some cosmetic edits, but that check still doesn't confirm whether the visible change made related to the error the bot intended to fix. Rjwilmsi 11:48, 5 January 2017 (UTC)

My (maybe limited) understanding of this case is that the "error rate" with regards to non-substantive saves was too high, with concerns about intended fixes not being made and inadequate edit summaries being secondary. Your fix to AWB (thank you) should reduce this error rate. Increased use of "skip only whitespace" could reduce it further. Better timing of article lists to work on before running the bot on them might help even more. What I'm left wondering is how much of an error-rate of non-substantive saves would BAG members tolerate. With the AWB fix becoming available and improved use practices adopted, maybe Yobot should be given a test period to see if the problems that bother editors are significantly reduced. Stevie is the man! ^{Talk • Work} 12:19, 5 January 2017 (UTC)

Closure?

@Ramaksoud2000: on the contrary I think it would be helpful if BAG could close this discussion sooner rather than later, as it is entirely within their purview and will probably streamline the arbitration case. — Martin (MSGJ · talk) 09:31, 5 January 2017 (UTC)

That's fine. I just didn't want people to continue to take the time to discuss something that may no longer be relevant soon. Ramaksoud2000 ^{(Talk to me)} 14:40, 5 January 2017 (UTC)

@Anomie, HighInBC, MBisanz, MusikAnimal, Slakr, The Earwig, and Xaosflux: Sorry to ping you all, but it would be helpful to have an official closure of the discussion above by a member of BAG. The discussion has been open more than a month and has not been edited in the last two weeks. Alternaitvely if you decline to close the discussion (for whatever reason) perhaps a short statement to that effect would be useful so the arbitrators of Wikipedia:Arbitration/Requests/Case/Magioladitis know where they stand. Thank you — Martin (MSGJ · talk) 14:06, 23 January 2017 (UTC)

Alternatively, ask for an uninvolved admin to close it at WP:ANB if no one from BAG is willing to do it. I don't think this can be left just hanging. SpinningSpark 14:13, 23 January 2017 (UTC)

It's not clear under current policy that admins can do that. If no BAG member will handle closing deauthorization discussions, we need to rethink how we handle bot approvals. Makes little sense to have a body who approves but won't withdraw approval. ~ Rob₁₃^Talk 14:20, 23 January 2017 (UTC)

The policy (WP:BOTAPPEAL) is quite vague on how this process should work. I think we should wait for BAG to close the discussion (or at least to indicate that they decline to act) before asking random admins. — Martin (MSGJ · talk) 14:22, 23 January 2017 (UTC)

Yes, BAG should make a statement, but that should not be to do nothing and leave it in limbo. My suggestion was that BAG themselves should ask for an outside closer if none of them feel able to do it. SpinningSpark 14:32, 23 January 2017 (UTC)

I'm speaking for myself that I don't specialize in evaluating consensus of large, contentious debates. Here we are discussing revoking all of Yobot's approved tasks. Those tasks were approved though, by BAG, so the thing we are supposed to specialize in was already done, and we're being asked to redo it. I sort of have my own opinions as well: some of these tasks seem helpful to me, but as some have said, they may no longer be supported by consensus. The consensus part I don't think you need BAG for. If there is a new consensus that an approved task should be terminated, it is my opinion that overrides any previous BAG approval, since consensus is a requirement for it to have been approved in the first place. This is no different than having a policy change based on new consensus after a discussion was closed by admin A, where you are amending an older policy change that was a result of discussion closed by admin B. Admin A does not really need to consult admin B, the community at large has changed their mind. So if it were up to me, I'd say any uninvolved admin or even experienced non-admin should be able to make the call here, especially since the issue seems to be closely tied to operator behaviour. I also think the outcome of this discussion should in some way be reflected at WP:BOTAPPEAL, as indeed this process isn't very clear — MusikAnimal ^talk 16:30, 23 January 2017 (UTC)

I've looked this over, as well as much of the discussion in the ArbCom case, but I'm not seeing which way to go.

The initial premise here is "Yobot's cosmetic edits are too much trouble, let's wipe out all of its tasks and start over", which is a lot of wiping-out when it comes to a bot with twenty-some approved requests. As far as I can tell no one has even tried to identify which of those tasks are still active or which tasks besides Yobot 16 are the source of the cosmetic edits, but several people have brought up that point.

Checkwiki fixes in general seem to have started to be controversial. Again, it would be helpful if someone would identify which checkwiki errors have cosmetic fixes and which are "substantial", and then identify which of the cosmetic fixes have a consensus to override WP:COSMETICBOT (in the community beyond the few editors who actually deal with checkwiki).

The ideal thing here, if I'm reading the underlying discussion currents correctly, would be to stop the tasks that are using AWB general fixes to do the actual work while allowing tasks that have functioning skip conditions to proceed. If a "general fixes" task can be fixed to have a functioning skip condition (versus the historical "I'll try to make better lists" non-fix), I wouldn't oppose restarting it. Yes, I realize that may take a fair bit of work to untangle the actual fixes from the other general fixes, but it's work that seems needed if these tasks are going to continue. I'd also require identifying the particular task being run in each edit summary, which should be easy enough to do. I'd also like to amend the checkwiki tasks (whichever those are) to only make the cosmetic fixes (whichever those are) when a non-cosmetic fix (whichever those are) is being made too, but I can't be more specific than that because this whole gigantic discussion doesn't have any specifics.

So, with all that vagueness here, I've fallen back to hoping the ArbCom case will provide something to make this discussion irrelevant. Anomie⚔ 05:48, 24 January 2017 (UTC)

@Anomie: Part of the problem worth mentioning is that Yobot has historically not included task numbers in edit summaries, so I'm unable to identify which tasks are actually causing the problematic edits. (As an aside, project tagging has been separately an issue with this. See, for instance, User_talk:Yobot/Archive_9#Stopping, where an admin nearly blocked Yobot for operating a tagging task against consensus. A lot of this stems from bot operator conduct, not the actual edits, which is part of why a complete revocation of the plethora of over-broad tasks is needed. If you look through these approvals, most of them would never be approved today due to the broad nature of them.) ~ Rob₁₃^Talk 05:53, 24 January 2017 (UTC)

@Anomie: thanks for looking and commenting. I agree there are a lot of considerations and it would be difficult to reach a conclusion with the discussion above. I believe (and it looks like you agree) that each individual task needs careful consideration (Is the task still active? Are there frequent cosmetic-only edits attached to this task? Do they have solid consensus?) and the best forum to undertake this analysis would be a separate BRFA for each task. This is why I am asking for revocation so that this can happen. Obviously inactive tasks will not need reapproval, so paring these tasks down should simplify the whole matter. — Martin (MSGJ · talk) 09:23, 24 January 2017 (UTC)

There is not a specific task causing the problematic edits. Last month it was error #104 that caused BOTH Dexbot and Yobot go wrong. The reason was that the list generated contained more pages that Yobot/Dexbot could handle or even should fix. All this misunderstanding started from there. Since the list was altered with pages that actually need fix, the issue should be over. At least for now. The last 2-3 months there have been some generalisations of the list in order to catch up with WPCleaner. We did them one by one. The comparison between the two lists can be shown in User:Magioladitis/AWB and CHECKWIKI. -- Magioladitis (talk) 08:32, 24 January 2017 (UTC)

I think the approach is wrong here. If someone wants a task to be disactivated, the should go to the CHECKWIKI page. For example, after the MEdiawiki pages errors 6/37 which are dealing with special characters in DEFAULTSORT where minimised to almost. After the removal of the interwikis from En.wp the same happened to all errors related to interwikis. The problem is that most people seem to find the tasks themselves useful for various reasons even not all of them change the visual output. From that point of vie we should distinguish the edits that fail to perform a task and the edits that do a task that we do not want to see it done as standalone. Because if we choose the first then we will see that Yobot apart from last month's fail, had very few fails in the last 2 years. The changes in the last 3 months results in some fails which were handled. Another example is error 69 (ISBN syntax fixes) which changed from a bot task to a manual task after the CHECKWIKI guys added more cases which actually need people to check them. ---Magioladitis (talk) 08:38, 24 January 2017 (UTC)