Archives

Archive 1 Archive 2 Archive 3 Archive 4 Archive 5 Archive 6 Archive 7 Archive 8 Archive 9 Archive 10 Archive 11 Archive 12 Archive 13 Archive 14

This page has archives. Sections older than 14 days may be automatically archived by Lowercase sigmabot III when more than 3 sections are present.

Hi everyone,

Following up on Jon's summary of the concerns, I'd like to explore some of the options we have for making the Wikidata descriptions better and more reliable.

To start with, the screenshot here is an example of how the descriptions are useful for navigation. This is the "top read" feature on the iOS Wikipedia app, which shows the articles with the highest pageviews for each day. I look at this every day; it's one of the ways that I keep up with what's happening on Wikipedia, and in the world.

In this case, I think 3 of the 5 top articles benefit from having a short description -- I have no idea who Gennady Golovkin and Canelo Álvarez are, and having them described as "Kazakhstani boxer" and "Mexican boxer" tells me whether I'm going to be interested in clicking on those. (The answer on that is no, I'm not really a boxing guy.) I know that Mother! is a 2017 film, but I'm sure there are lots of people who would find that article title completely baffling without the description. Clicking through to the full list of top read articles, there are a lot of names that people wouldn't know -- Amber Tamblyn, Arjan Singh, Goran Dragić. This is a really popular feature on the apps, and it would be kind of useless without the descriptions.

That being said -- I've also occasionally been annoyed and mildly embarrassed by a description. I read the Wikipedia article on Evil clown on the app a while back, and the description said "Evil clowns are evil clowns that are scary" -- an obviously juvenile description. When I got to my laptop, I went to Wikidata under my volunteer account, and changed the description to "Pop culture trope". But I only knew how to do that because I work at Wikimedia. If I was still a Wikipedia volunteer contributor who didn't go to wiki conferences, I don't know if I would have heard of Wikidata, and I wouldn't know how to fix this.

So I think it's important to find a way to keep the usefulness of the short descriptions, and improve the quality and reliability. There have been several different options proposed here, and I want to summarize them, look at some pros and cons, and then talk with everyone here about changes that the product team/communities can work on to make this better. I'm sure that I'll miss some stuff, so it would be great if folks could help me fill in the gaps.

1). Having more eyes on the descriptions, encouraging people to fix mistakes

This is one of the goals for allowing people to edit descriptions from the Android app. If there are more good-faith people looking at descriptions and it's easier to edit them, then vandalism gets reverted faster, and inadequate descriptions are replaced by better ones.

The potential con: it's also easier for bad-faith people to edit the descriptions, and we're not sure how quickly those would get picked up and reverted/corrected. This is also true for well-meaning people making "bad edits", as Jytdog points out in the Vandalism on Wikidata discussion above; these might not be picked up as vandalism, but need to be corrected all the same.

I'm not totally familiar with how the moderation works on Wikidata; hopefully folks can help me out here. Are there enough people watching items to patrol descriptions effectively? When the apps team rolled out description editing on the Russian, Hebrew and Catalan apps, the data they collected shows that 4.6% of the description edits were reverted. This will need to scale to English.

I see a recent discussion on the Wikidata:Project chat -- "Pending changes?" -- where people are acknowledging that vandalism is an issue, and suggesting either a pending changes system or using ORES or bots to surface likely vandalism on items that have Wikipedia pages. That's potentially an area that we could support.

2). Including Wikidata description edits in Wikipedia recent changes, watchlists, history

Surfacing Wikidata description edits to Wikipedia editors would provide more experienced eyes who could look at the edits, and make sure the descriptions are appropriate and accurate.

The Wikidata team is currently working on separating out the relevant changes to surface -- in this case, just English descriptions. They're just about to deploy a test version on Greek Wikipedia to see how their current solution scales.

There are some questions about this approach: Is this going to create extra work that some people don't want to deal with? Is it a barrier if you have to go to Wikidata to revert? What if you get in a revert war with someone, do you have to go to Wikidata to report and resolve the situation?

There would be further feature work depending on the answers to those questions. It would be helpful if people patrolling description changes on Wikipedia could do as much from Wikipedia as possible, rather than switching over to Wikidata.

3). Magic words in Wikipedia articles to override Wikidata descriptions

This could be a magic word like {{DESCRIPTION: .

This would be a more direct way for Wikipedia editors to ensure that moderation or discussion about a specific page's description happens on Wikipedia, and not on Wikidata. It would only replace the Wikidata description when there's something typed in the field, not just blanking the description completely.

A con for this idea is that it might be confusing for people who see a bad description and want to change it -- do they go to the Wikipedia page, or the Wikidata page? Also, this would have to be accounted for in the app description-editing feature.

4). Use the first sentence of the Wikipedia article, when appropriate

This was proposed and discussed above, including a comparison table with sample descriptions. It would have to filter out some pieces -- "(title) is a", birth and death dates, translation and pronunciation, etc.

Pros for this approach: it's derived from Wikipedia text, so it's under enwiki control. Also, it guarantees that there is a description; there are gaps in the Wikidata descriptions, but every article has a lead paragraph.

A con for this is that sometimes the first sentence is too long or dense to be helpful as a short description. Maybe we can come up with an idea for how to handle long first sentences. The Wikidata descriptions could be a fallback, or the magic word.

5). Protecting the description on protected pages

This would protect the short descriptions on pages that are likely vandalism targets, presumably synching protection on Wikidata with protection on English Wikipedia, so that admins protecting a WP page wouldn't have to remember to do something on Wikidata.

I'm not sure exactly how this would work, and can't find a description of it right now. Have people figured out a mechanism for this? How would you go about protecting the description on Wikidata?

So there's a lot of questions, and I'm sure I missed some pieces, but I'm hoping that this helps focus some discussion on potential solutions. What do you think? -- DannyH (WMF) (talk) 22:03, 19 September 2017 (UTC)[reply]

My preference would be a combination of 3 and 4: Use a local magic word when available, fall back to (a filtered version of) the first sentence otherwise. That way everything is under local control, the interaction with Wikidata is significantly less complex, and (unlike Wikidata) we're guaranteed to get something. The descriptions will occasionally be too long, but that's not really a worse problem than being occasionally missing, and if editors insist on long sentences in article text we can still achieve tighter descriptions using the magic word. If Wikidata wants to run bots that import this information from Wikipedia to use as their descriptions, they could also do that, but it would be their local decision and wouldn't affect what we see here. —David Eppstein (talk) 22:09, 19 September 2017 (UTC)[reply]

This is not place or time to solve this problem in my view. This is (again) all begging the questions. To put a pointy analogy on it:

Putin: "So let's discuss the plans for the highway system in Crimea. There are lots of problems, lets fix them!"

Rest of world: "Dude, what the hell are you doing in Crimea"

The questions are 1) about WMF's role in making content decisions, and 2) even deeper questions about navigating differences in the various projects' content policies and governance processes.

I am not convinced that folks from the WMF even understand these two fundamental issues, and "solutions" that aren't grounded on a good understanding are also going to be unstable.

Reversing direction, and stepping back, is the correct way to go here. Not deeper in.

Please reverse course. Jytdog (talk) 23:49, 19 September 2017 (UTC)[reply]

Having descriptions (on mobile, on Visual Editor, etc.) is an unambiguous good for Wikipedia. As a Wikipedian, I have zero interest in this boundary conflict, and think the analogy to sovereign independent states is pointless. We are neither sovereign nor independent, we are collaborating and interdependent. Moreover, this is a list of options, (almost all of which) give Wikipedians control over "their territory": how things look on Wikipedia. Let's skip the diplomatic crisis and figure out how to make this work best.

I tend to think that Options 4 & 5 are strong starting points. I also would love to have a tool that shows the descriptions for my watchlist, and either lets me edit those descriptions or has clickable links to the corresponding Wikidata pages. (Editing description on Wikidata is actually very easy, unlike a lot of things over there.)

The real questions are: can we develop a manual of style for descriptions? Is that MoS compatible with Wikidata expectations? And, how does the Wikidata community feel about mass importing new descriptions from Wikipedia?--Carwil (talk) 01:37, 20 September 2017 (UTC)[reply]

It was a pointy example as I pointedly said, and trying to drive that down my throat is bullshit. I offered the colorful example because folks from the WMF don't seem to be getting it, that they overstepped and did a fucked up thing after they overstepped, and that continually setting the focus of the conversation on fixing their fuckup is not legitimate (it is as weird as talking with putin about fixing highways in Crimea... the whole conversation is surreal and begs the question - namely - why are we having this conversation at all?)

Collaboration depends on recognizing appropriate roles for all the collaborators. As Rilke said, "love is respecting the difference between". The WMF overstepped its role and its competence by doing this unilaterally (by 'competence' i mean that it, as an organization, didn't understand the policy and governance issues around this content).

The wrong thing to do now is flail around trying to fix this multi-layered fuckup. The right thing to do now is undo, step back, and think clearly before trying to implement again. Jytdog (talk) 01:52, 20 September 2017 (UTC)[reply]

Let me amend this Crimea analogy… The infrastructure contractors for Kyiv have built installed a subway system (mobile app, mobile view, etc.) that lets millions of people get around our city. In the process, they have installed navigation signage that draws on Russian-language text. What the WMF people are saying is: hey, how would you like us to get text for the signs. In response, what Jytdog is arguing is that the contractors should apologize for putting up signs at all. If your beef with WMF is so important to you, Jytdog, fine. Just stop pretending you speak for all of us.--Carwil (talk) 04:13, 24 September 2017 (UTC)[reply]

The community already rejected these roadsigns, installed by the WMF, for the mobile app. I am not speaking for myself. Jytdog (talk) 04:33, 24 September 2017 (UTC)[reply]

DannyH, the only acceptable options are those that are enwiki based (for enwiki articles). Descriptions are textual, language-based content which should reflect the article, and should be embedded and maintained by the project and editors, not by another project with completely different goals. The best solution seems to be some template placed at the top of the article. If this is easier for the WMF, we should strive to gove it a "universal" name, so that it can be used on all wiki-languages that want it. Initially, it can be filled by bot either by copying the Wikidata description or by a shortened, cleaned version of the first line of the article, whichever people think is best (I prefer the second, but this bit is not essential to the proposal). From then on, this gets maintained locally, independently of the Wikidata descriptions.

You say "A con for this idea is that it might be confusing for people who see a bad description and want to change it -- do they go to the Wikipedia page, or the Wikidata page?" which is obviously false. This would be simply a part of the article, very few people would really be confused and think "oh, I need to go for Wikidata for this". Why would they ever think this in the above solution? Fram (talk) 07:15, 20 September 2017 (UTC)[reply]

• It's quite a task we're looking at. Five and a half million short descriptions is a hell of a lot of text to create and maintain. Over on Wikidata, I've been working on items for civil parishes in England. I hadn't got as far as the descriptions yet -- first I have been working on tidying up data and building up external matches. The prospect of addressing the descriptions has been quite daunting, even with only 15,000 items of more or less the same kind to consider -- a drop in the ocean compared to short descriptions for all the articles on en-wiki. (And I have to say, I am very dubious that one could simply automatically get the information from article first lines.

I get that there's a lot of unhappiness about Wikidata here, but one (admittedly small) thing in the other side of the scale is that at least it makes it possible to a quick idea of what sorts of descriptions there are, and also to (relatively easily) make systematic edits to large numbers of them at scale.

Anyway, to get an idea of what sorts of things we might like (and not like) for descriptions, it's maybe worth looking at some samples of what Wikidata is currently providing. In each case, the link below goes to a query for 200 examples -- follow the link and click the blue button half way down the left hand side to run the query.

• People: tinyurl.com/yd6dvub7
• Castles: tinyurl.com/ybxdr7g5
• Populated places: tinyurl.com/yc3yq6gf
• Rivers: tinyurl.com/ybdh6blw
• Albums: tinyurl.com/yatcou33
• Films: tinyurl.com/ybkk9928
• Unidentified things: tinyurl.com/y7k3w6ud
• Civil parishes in West Sussex: tinyurl.com/ydd2utow

(Note: to see more examples, increase the number in the LIMIT clause; to see different examples, change the number in the OFFSET clause; I didn't force a sort order, in order to increase variety, so the same query may produce a different set of 200 examples if you run it in 30 minutes time -- for a deterministic query, to get a repeatable set, add eg ORDER BY ?item before the OFFSET keyword. The 'link' button on the left hand side can be used to get a short URL to the modified query).

It is apparent that there is no great consistency or house style. (Though even what is there already may be useful for some purposes -- eg disambiguating searches). This is one question it would be really useful to have input on, eg from the reading team: what is actually wanted? I imagine there is quite a lot of flexibility in what is useful, but what is the ideal? How detailed/verbose is useful? How brief/succinct is better?

So for example for a civil parish like Donnington (Q666080), we currently have "village and civil parish in the Chichester district of West Sussex in United Kingdom". Is this appropriate? Would the shorter "village and civil parish in Chichester, West Sussex, England" be better to standardise on? Or drop Chichester district, and just "village and civil parish in West Sussex, England" ? Or just "civil parish in Sussex"? All of the above could be systematically rolled out, but what is most useful?

Looking at some of the other result sets, it appears that a lot of film descriptions have been standardised to "<year> film by <director>", sometimes with genre also given, or occasionally country of origin; a few have more discursive descriptions (eg Oliver Twist (Q645752), Thunderpants (Q708825).

Descriptions for albums seem less systematised. Most common seems to be "album by <artist>", sometimes specifying whether it is a studio, live, or compilation album. Some are longer, eg "debut studio album by the American rock band Flowerhead" (...Ka-Bloom! (Q14954705)) or "fourth studio album by CCM singer Mandisa" (Overcomer (Q15078867)). Many just say "live album" or "album".

For people a common form seems to be "<nationality> <occupation>". (I think these may have come from the old PERSONDATA templates -- more recently created items may not be so systematic). Again, some items are more discursive, eg Banksy (Q133600). Is this useful?

Raising quality and usefulness for short descriptions, wherever it happens, is going to be no small task. But a useful thing, however we go forward, would be some more discussion on what is considered a useful description, and what is one that could be improved. Jheald (talk) 12:36, 20 September 2017 (UTC)[reply]

I agree with Jytdog that this feature was implemented a couple years ago without due care and attention to the differences between the two sites. I don't think that's catastrophic; it's a mistake that we have to fix. But what we have right now is a feature that's useful, and people like it. I don't think that breaking the existing feature is a necessary step before we figure out how to make it better. That's basically punishing the people who use the app, in order to make a point that they don't know or care about.

Fram, I agree that creating a completely enwiki-based alternative is one of the options. I mentioned the problem of not knowing where to fix the description, because I was thinking of it as a combined system that used Wikidata as a fallback. As Jheald says, a con for the completely enwiki template/magic-word version is that there would be five and a half million descriptions to generate, and that's a serious thing to think about.

Jheald, it hadn't occurred to me that this was an opportunity to standardize the descriptions. That's actually an important thing to figure out, no matter where the descriptions live. I need to read your post again, slower, and think about it. I'm really glad you brought it up. -- DannyH (WMF) (talk) 14:21, 20 September 2017 (UTC)[reply]

Considering that Wikidata was apparently able to reliably source 50 million statements in a month (a feat no one has been willing or able to explain so far, but which is shown in the statistics and pushed Wikidata over the "50% of our statements are reliably sourced" threshold suddenly), it shouldn't be too hard to add 5 miilion descriptions to enwiki in a reasonable timeframe, if this is the way we choose (some clever programming which parses and "summarizes" the first sentence may also be an option and then wouldn't need article edits). We need a) a decision of the best solution (a global template, an enwiki template, or clever reading of the first line at runtime?), b) a decision (in the case of a template) of how we would populate it initially (copy the wikidata description, clever reading of the first line, something infobox or category related, ...?) and c) a botrun. In the meantime, we can decide whether we ask the WMF to turn off the Wikidata descriptions display off immediately, or wait until the bot run (or other solution) is finished.

If we start by copying the Wikidata descriptions, we don't even need to edit 5 million articles, as many Wikidata items don't have descriptions at the moment. (First random item I encountered, [1] doesn't have one and doesn't need one). Fram (talk) 14:41, 20 September 2017 (UTC)[reply]

Fram, I want to make sure that some other people get a chance to talk before we jump straight into a solution. I'm not trying to backtrack, I just want to make sure people can talk about the options.

If it turns out that's the solution we go with, it would be good to include the product team in the implementation, not just run a bot and ask the WMF to turn something off. We're offering to be a part of this, and work on this collaboratively. One thought about switching to Wikipedia-populated descriptions: we could think about porting the changed versions back to Wikidata, to be the English descriptions there. I know that not everybody here is interested in improving Wikidata as well, but it would be good to find a solution that benefits everyone. -- DannyH (WMF) (talk) 18:21, 20 September 2017 (UTC)[reply]

I can only speak for myself (I may sometimes give another impression though ;-) ), but I see no reason to not take some time to find the best solution here, and have no problem with a solution which then takes some time to actually implement. I have indicated a few times what I believe are minimum requirements for such a solution, but it makes no sense to e.g. create a Template:ShortDescription here when perhaps the WMF would prefer a somewhat different Template:Summary or a magic word or whatever instead, which they can then read more easily (to use in apps, search, ...) and which may be something other wikis can implement without too much effort if they want to. We have the time to do this right, and we have the time to run a massive botrun if that is required. As for your final point, I see no reason why Wikidata couldn't import (or mirror) our descriptions if they then would like this: it's not something we can or want to impose, but it would be nice. Fram (talk) 19:09, 20 September 2017 (UTC)[reply]

My views on these options (from a perspective that using the Wikidata descriptions are a good idea in principle) are:

1. Making things easier to edit is always a good step forward, however there should also be an easy way to revert the edit. Maybe some sort of "this was recently changed, does it look bad?" tag may be useful there.
2. Sounds good, but it would be better to make all Wikidata edit summaries more useful rather than picking on particular bits.
3. This is a terrible idea - it's basically duplicating a bit of Wikidata here (and, presumably, also on every other language Wikipedia). It would be much better to have some way of editing the Wikidata descriptions directly here instead.
4. Sounds good. Maybe this could be done in such a way that it can feed into Wikidata - either by a bot, or some sort of gamification/user checking. I think this would be welcome on Wikidata, particularly for entries that don't yet have English descriptions.
5. This is something that needs properly thinking about by people that are active with page protection. If a page is protected here, should the wikidata entry as a whole be protected, or all of the wikidata properties used in the page? Or some sort of semi-protection? Although we don't currently do this with Commons media included on a page.

Thanks. Mike Peel (talk) 23:08, 20 September 2017 (UTC)[reply]

@Mike Peel: I see your point about a magic word here being redundant duplication of what is already on Wikidata but I consider it that Wikidata is itself duplicating a role that should be en's (describing things in English text) rather than vice versa. I would only be willing to see Wikidata descriptions used here if we can make them subject to our local standards for sourcing, etc. Consider e.g. the long debates over whether someone should be described as a "climate change denier". We have well-established rules for sourcing biographies of living people here, despite which these questions are still difficult. Should we let tendentious editors forum-shop Wikidata's much looser rules over ours in such cases? I think not. —David Eppstein (talk) 23:20, 20 September 2017 (UTC)[reply]

@David Eppstein: I care more about avoiding the duplication of work/content than I do about which project should have which role. Technically, I think Wikidata has better infrastructure to hold the information (structured rather than free-form content), while the community and English-specific expertise is here. If we could forget the different project names, the ideal case might be the enwp community taking ownership of the English Wikidata descriptions and running them according to enwp rules. ;-) Thanks. Mike Peel (talk) 23:30, 20 September 2017 (UTC)[reply]

Since when are short bits of text "structured rather than free form content"? Fram (talk) 06:50, 21 September 2017 (UTC)[reply]

For me (4). Use the first sentence of the Wikipedia article, when appropriate is the only one that makes sense. The con is that "sometimes the first sentence is too long or dense to be helpful as a short description" but this is not a problem for Wikidata descriptions but for Wikipedia. Our guidance, at WP:BEGIN, is that "If possible, the page title should be the subject of the first sentence", and "Try to not overload the first sentence". The first sentence, as part of the first section, should also be accessible, even in a highly technical article. So too long, dense, or technical first sentences are not appropriate for WP.

The only issue is what to do when the first sentence does not describe the topic. E.g. List of songs recorded by Steps (from tomorrow’s front page) begins "The British group Steps have recorded songs for five studio albums". Wikidata though describes it as a "Wikimedia list article"; presumably other lists can be handled the same way. Other topics that are hard to describe might use a standard form, or just the title, as e.g. History of Iran does.--JohnBlackburne^words_deeds 23:34, 20 September 2017 (UTC)[reply]

"Wikimedia list article" is a rather dreadful description. That article doesn't even need a description, the title is self-explanatory, but if it has one it should be something "Songs recorded by Britsh band Steps", i.e. a description which adds something useful to the title, not a description which adds a totally irrelevant concept (Wikimedia) which can only confuse people instead of helping them. It looks as if all List articles have the very same description at Wikidata, which is again a good argument to take this functionality away from there and give it back to the local wikipedia versions (at least those that want this). Fram (talk) 06:50, 21 September 2017 (UTC)[reply]

(Of course, there may be a very good reason why these list descriptions are ideal for Wikidata, and I am not arguing that they should change them; but for enwiki, they are just not useful at all) Fram (talk) 06:55, 21 September 2017 (UTC)[reply]

The list description needs to be considered in the context of how it would be seen -- namely in a list on a mobile screen as in the picture shown at the top of this article, or as an explanatory line in a set of search results. In those contexts, I submit, it is quite adequate. Jheald (talk) 09:16, 21 September 2017 (UTC)[reply]

Umm, how? They explain nothing at all, they only introduce a concept which has no useful information and proably no meaning for the average reader, "Wikimedia". Thta it is a "list article" is rather obvious from the name anyway, so the end result is two superfluous words and one irrelevant one. In what world is that "quite adequate"? Fram (talk) 09:21, 21 September 2017 (UTC)[reply]

I'm gonna suggest an alternative: Let's just take the first sentence from TextExtracts. Just for English Wikipedia. Problem solved, little engineering required, English gets to be holier than holy, rest of the wiki's get the benefit from centralization and structured info, apps retain 'some form' of disambiguation labels for en.wp. No need for specialized edit interfaces, no need for magic words. Just toss ellipsis at the end, and maybe reduce the font size a bit and just live with that. I think first sentence is subpar from the descriptions that Wikidata provides, but we have to find some place of compromise and this seems like the least terrible option to me. I definitely DONT see a reason to spend engineering effort to duplicate Wikidata descriptions in Wikipedia.—TheDJ (talk • contribs) 07:30, 21 September 2017 (UTC)[reply]

What exactly is the benefit of centralizing descriptions, which are language-based? This means that you need editors fluent in all these languages to not only patrol and edit their own language wikipedia, but also Wikidata. Apart from that, you seem to continue to repeat the same misconceptions about "structured data" when it is just a short freetext sentence fragment, and the supposed superiority of current Wikidata descriptions when just above it is e.g. shown how all lists have an utterly useless description at Wikidata: we have about 100,000 of these, with many of them among our most popular articles, so this is not something marginal. To push for a subpar, English-only solution out of some kind of bitterness is not productive or helpful, when there may be opportunities here to make things better for enwiki and other wikis that might prefer the same solution in the end. Fram (talk) 07:56, 21 September 2017 (UTC)[reply]

A Wikipedia centric approach. From the global state of information and knowledge... what is it that that list is ? It is nothing physical in the real world that matches it from a data topic point of view. It's a Wikipedia article. Now if it were called "Discography of Steps", then that description might be different, because a discography is a real life concept, but 'list of' has no inherent meaning for most people.

When people say structured, it means structured storage and linking, not structured authoring. While for one property the value of that might be limited, when combined with all the other properties it is a very strong and Wikidata is proving that again and again. Wikidata is becoming the entry point for how others interact with the content in our movement, Wikipedia is the long form appendix that you can link to when someone is really interested. Making things better for Wikipedia is awesome, but I think that for 95% of all our language variants for Wikipedia, this change would have 0 benefit and might even be detrimental. Even for English Wikipedia the quality concerns are limited to a very specific set of articles. And that forgoes the point that I think that the ROI simply doesn't make sense for that option. Prediction; will take a year to complete such a feature and only 5-10 wiki's will use it. There is no space for it right now in the roadmap, and no budget allocation either... Shortcuts are the only option that I think are serious candidates. —TheDJ (talk • contribs) 09:24, 21 September 2017 (UTC)[reply]

Are you really arguing that in general, people will know what "Discography of X" means but will have trouble understanding what "List of songs by X" means (unlikely), and that somehow "Wikimedia list article" will then somehow lead to a better understanding for these people? I'll try to stay polite and just say that I disagree and think that most uninvolved people would disagree as well. As for your arguments about structured data, the descriptions are one bit which is actually quite separate from the remainder of Wikidata and doesn't really benefit or interact with the remainder. It just doesn't fit in with what Wikidata is and how it is structured at all, it is presented differently, edited differently, ... It has no room for sourcing, it is multilingual, it is everything the remainder of Wikidata is not. As for roadmap and budget problems, fuck them. Enough time and budget has been spent on things no one wanted or needed, and a lot of time has been spent by volunteers to convince the WMF that some things really are not welcome and should be shut down or totally rewritten. Here they have the chance to work together instead of against the community, and to develop a solution which may be beneficial to everyone involved. Fram (talk) 09:56, 21 September 2017 (UTC)[reply]

There is actually a problem here. Indeed in 99.99%, or most likely in 100%, "Discography of X" means "List of albums/songs performed by X". However, once in 00.01% cases (which may not apply specifically to Discography but may apply to other examples) it can make a name of a magazine, or a music band, or indeed a discography of another performer with a similar name. This is why it happens to be useful. Note that whether there are other meanings or not are language specific.--Ymblanter (talk) 10:11, 21 September 2017 (UTC)[reply]

For clarity: this shows that currently only one (Discography of Nico Carstens) out of 11 non-redirect article titles in the "Discography of X" format is an article containing a "List of albums/songs performed by X" (so less than 10%, with only one single instance, instead of "99.99%, or most likely in 100%"). --Francis Schonken (talk) 10:27, 21 September 2017 (UTC)[reply]

"This is why it happens to be useful. Note that whether there are other meanings or not are language specific." Your conclusions don't follow from your reasoning. Are you arguing that a generic "Wiimedia list article" is useful because not all "Discography of X" articles are a list of recordings of X? That's a total non sequitur. As for "language-specific", yes, but how is that relevant? Perhaps the Finnish equicvalent of "Wikimedia list article" is extemely helpful in Finnish, fine, then use that. But people who are so hell-bent on keeping the Wikidata descriptions that they can't even admit that 100,000 or so instances of "Wikimedia list article" are 100,000 examples of totally useless Wikidata descriptions (in English) for use on enwiki views, searches, ... simply make a mockery of this discussion and any attempts to find a rational compromise. Fram (talk) 11:55, 21 September 2017 (UTC)[reply]

First, I hope I demonstrated that your statement that descriptions are not needed because every sane person can say from the title what the article is about is wrong. Second, a "Wikimedia list article" description is not ideal, but it is better than no description, because is says, well, that the article is a Wikimedia list article.

I said that for most list articles (like list of Steps records) no description is really necessary, and you demonstrate something (not really sure what) about "discography of " articles, which are not the same (and don't even have a "Wikimedia list article" description by default. So no, you haven't demonstrated anything about my statements. Further, please explain in what way "Wikimedia list article" is better than no description? That it is a list article is obvious as it is an article titled list of; only the exceptions, where an article with such a title in fact isn't a list but e.g. the title of a book, film, or record really need a description. Like I also said already, even if they don't need a description, it may be helpful to have one which gives the reader more useful information, e.g. describing "List of Steps recordings" as "Recordings by British band Steps" or something similar, which adds that they are a British band, and not e.g. a type of dance. The current description though, adored now already by three of the more vocal defenders of getting descriptions from Wikidata, doesn't add anything useful. Please, all of you, understand that either readers don't know what Wikimedia is, and will think probably it a typo for Wikipedia (and whether it says "Wikimedia list article" or "Wikipedia list article", or they do know what Wikimedia is, i.e. the foundation behind the site they are surfing on. It tells you absolutely nothing, nada, zilch, about the list of X you are searching for or just started reading. It is just an annoying "thing" you get at the top of every list and which you have to scroll past. Having no description is in these cases clearly better than having this generic, bot generated description. Just like it makes no sense that our 200,000+ disambiguation pages like Blackboard (disambiguation) get the description "Wikipedia disambiguation page" (oh, here it is Wikipedia and not Wikimedia?); no kidding, Sherlock. (Did you know that we have a page titled Disambiguation (disambiguation)? The Wikidata description is "Wikimedia disambiguation page", which I wouldn't have known otherwise). Those descriptions make sense on Wikidata, where e.g. this doesn't have (or need) "Disambiguation" in the title: they don't make sense on enwiki. The fact that they serve two different purposes, two different needs which require two different descriptions is a very good reason why this feature should be hosted on enwiki for display in enwiki searches and articles. Fram (talk) 12:53, 21 September 2017 (UTC)[reply]

Honestly Fram, is this really the best you can do? So some of the descriptions are mildly redundant. Is that really such a big deal? Seems "mostly harmless" to me. I thought you had some concerns that were actually *serious* about WD descriptions. Yet I post some queries so we can look at typical samples, and you're not even bothered to comment on them. Get serious or stop wasting everybody's time. Jheald (talk) 13:27, 21 September 2017 (UTC)[reply]

No, this is not "the best I can do", it is but one argument among the many I have made. I try not to repeat everything in every post, as I already get the complaint of carrying on a monologue (which, considering how few of the replies I got had anything to do with what I actually said, is probably accurate). My main reasons are that a) the WMF should never push data from another site into enwiki content, b) this is language-based content which is not the domain of Wikidata but of the different language-based Wikipedias, c) Wikidata policies are too different (or lacking) compared to enwiki d) vandalism reversal at Wiki is on average way too slow (and note that while statistically vandalism is rare at Wikidata, in reality for descriptions it is pretty high), e) things like blocking, protection, ... on enwiki don't work on this, so people we want to get rid off can still subvert our articles through Wikidata (and Commons, but having one backdoor is hardly an argument to open a second one), f) hundreds of thousands of Wikidata descriptions are useless and confusing, and we would be better of without them, and g) probably a few other things I already said but which I don't immediately recall now... Which you would have known if you had actually read my posts, instead of taking pot shots at "is this really the best you can do" when I just gave yet another reason why this is a problem, not the only one or necessarily the best one. Anyway, looking at your queries, taking the one for albums, I see that the vast majority of albums have no description on Wikidata, and that for the remainder, about half is useful (giving extra, relevant information) and half is basically a repeat of the enwiki disambiguator we have in the title anyway (e.g. for "Spirit", it is nice to have the description "Depeche Mode album", but considering that our article is called Spirit (Depeche Mode album), it is hardly helpful to have this description). For your query for "unidentified things", only 11 of the 200 items (with an enwiki article) even have a description... Fram (talk) 13:50, 21 September 2017 (UTC)[reply]

For items with no description, it's worth noting the mobile app can fill in with an auto-description generated on the fly based on the wikidata statements available. Unfortunately I don't have a link to the code (perhaps somebody on this board can supply it?), but auto-descriptions by Magnus can be seen in action on eg Mix'n'match, which shows both auto-descriptions and manual ones for matched entries. Indeed, not two weeks goes by without User:GerardM suggesting that manual descriptions should be abandoned altogether, and descriptions just left to the automatic generator. I wouldn't go as far as that, as I think there are times when manual description can be necessary for clarification (eg two civil parishes with the same name in the same district of the same county -- yes, there are some examples of this). Also there can be advantages for queries in having baked-in text available. (Though in principle one could also make the auto-generated descriptions available within queries via a SERVICE routine).

But I am glad if discussion can take a harder look at some of the descriptions we have, and how they could be improved. External services and Wikidata-based systems will go on using this descriptions whatever WMF does or doesn't do, sometimes even as an index to Wikipedia content, simply because they are so accessible, so it's worth looking at them hard and thinking how we could do better. For the moment I think what we have at the moment is mostly better than nothing, and even in less useful cases mostly harmless; so I don't see it as a case of the sky falling.

I do share some concern about potential vandalism. It's one thing to say Wikidata's vandalism rate is low because it mostly skates under the radar as a low-reward proposition for vandalism. But much higher visibility and editability effectively paints a target on those statements. (Indeed, when this change was announced at Wikidata's Project Chat, my first instinct was to ask whether vandal fighters on en-wiki felt confident that they had the capability and the capacity to handle it. I think now after all the discussion above everyone would have to accept that the only current answer to that question has to be: No). But I think it's probably fixable.

As for BLP issues I think we need to accept that there is certain information in descriptions that we need to be very careful about, that perhaps should only be accepted on the basis of flagged revisions (perhaps triggered by particular words), if those descriptions are going to be presented alongside Wikipedia text. Giving Wikipedia admins the ability to activate flagged revisions for en-descriptions of a particular item (perhaps automatically if the item is protected on en-wiki) might also be a way forward to deal with concerns in that area.

Verifiability of descriptions (at least as interpreted as a requirement for explicit sourcing) I see as less of a concern, just as lead sections of articles on en-wiki don't require explicit footnotes, so long as they are supported by the rest of the article. Even with references, it's hard to control against people changing the information while leaving the reference. But for the most point, the contents of the description will be the most basic facts about an item, and are likely to be readily supported by the content of external links, and/or references in the attached wiki article.

In my view there is work to do on both the current description data and the underlying systems tying Wikidata and the wikipedias. But I believe this is achievable, to a good outcome. Jheald (talk) 14:52, 21 September 2017 (UTC)[reply]

I did not get the impression that you listen to what I am saying. I can reply, but, to be honest, I do not see any point in doing so, since you will likely come up with a new explanation completely unrelated to my reply, so I better do not. As I said earlier, I do not have any stakes in descriptions.--Ymblanter (talk) 13:07, 21 September 2017 (UTC)[reply]

I directly replied to the two statements in your post (conveniently labeled "first" and "second"). You are free to leave the conversation, but it would do you credit if you didn't fabricate reasons for doing so. Then again, you didn't refrain from fabricating things I supposedly had claimed ("your statement that descriptions are not needed because every sane person can say from the title what the article is about"), so this shouldn't come as a surprise. Fram (talk) 13:20, 21 September 2017 (UTC)[reply]

I can not really leave the conversation because there never have been one - you consistently fail to address the points I made instead cherry-picking quotes and making a strawman. As I said, I do not see any sense in continuing this exercise. Your position is also clear, it did not shaft a bit on this page irrespectively on any arguments which were brought by different users here. At some point, an RfC will be organized, and then we are going to see how much the community shares your ideas.--Ymblanter (talk) 13:28, 21 September 2017 (UTC)[reply]

We are here because an RfC was organized but only implemented partially by the WMF, remember? Fram (talk) 13:50, 21 September 2017 (UTC)[reply]

(ec) In most cases I have to say that I think just using mw:Extension:TextExtracts would not be very helpful. If we consider the image of the mobile screen at the top of this section, the short descriptions need to come to the point very quickly, without prefatory matter like pronunciation, etymology, field of the subject, or repetition of the subject name.

Unlike some others in this discussion, I am also pretty dubious about the extent to which good short descriptions could be extracted automatically. I think often they would still be too wordy and not suitable.

The point about whether it makes sense for WMF to put much in the way of limited engineering resources into duplicating delivery mechanisms when Wikidata already exists is a fair one.

@Fram: You are correct that the text slugs on their own are not structured data. However, it makes sense to put them into a retrieval system that is structured, so that as per the queries above they are instantly retrievable for any subset. As the queries above show, it's a lot easier to extract from Wikidata descriptions for eg all the civil parishes in England that it would be from eg Wikipedia -- it makes sense to store the descriptions in a structure that makes them directly accessible. Primarily these are going to be delivered as 'explanation lines' accompanying search results. One doesn't want to have to scrape the pages every time one wants to present the summary explanation line. Jheald (talk) 09:36, 21 September 2017 (UTC)[reply]

"Primarily these are going to be delivered as 'explanation lines' accompanying search results. One doesn't want to have to scrape the pages every time one wants to present the summary explanation line." Such scraping is done every time one does a standard search, e.g. this, where every result shows the start of the article anyway. Fram (talk) 09:56, 21 September 2017 (UTC)[reply]

DJ you want to say that Wikipedia is some kind of spoiled princess. Whatever. Here is a what a Wikidata editor wrote at Jimbo's If a change occurs to the Wikidata item linked to a Wikipedia page I watch, then that change shows up with a diff link to Wikidata. If you have issues with Wikidata's content, then you need to work with our (Wikidata's) community to work out a solution. Keep in mind though that Wikidata is a highly multilingual, mixed community where insisting that every Wikipedia policy be applied there is highly frowned upon.

No surprise to me, that a Wikidata person expects the norms of that editing community to be respected. The projects are different. There is this continued fantasy that norms are the same. They aren't. Jytdog (talk) 08:18, 21 September 2017 (UTC)[reply]

And maybe it deserves to be a spoiled princess. Maybe. But at the same time, we should also recognise that it is the exception. So instead of making everything follow Wikipedia's norms, the better option might be to have Wikipedia be the odd kid out. Princesses might grow up a bit in isolation at some point due to their status. —TheDJ (talk • contribs) 09:24, 21 September 2017 (UTC)[reply]

You continue with the sloppy garbage that it is OK for the WMF to play content DJ, breaking the fundamental deal here as well as policies in one or more projects, and setting projects against one another, in order to serve some aim that it sees as For The Greater Good. I am not going to get sucked into trying to convince people in Wikidata to follow en-wp BLP. That would be stupid and imperialistic. Nor am I am going to accept a mechanized process to introduce content into en-WP that is created without V or BLP. That too is idiocy. Jytdog (talk) 09:40, 21 September 2017 (UTC)[reply]

Could you please expand which people are going to be blocked and banned here? And by whom?--Ymblanter (talk) 09:49, 21 September 2017 (UTC)[reply]

Last week I had launched an RfC at VPP calling for blocks for the people who kept this going after we had the RfC in March, as this violated clearly expressed community consensus. I withdrew it when dialogue was opened last week. If the "dialogue" continues to beg the question on a "Lets fix the highways in Crimea" basis (see above in this thread) then I will re-open it. That one didn't specify individuals. Jytdog (talk) 10:13, 21 September 2017 (UTC)[reply]

Ok, thanks. You should have been more clear: not "people are going to be blocked or banned" but "I would like to see these people blocked or banned". The RfC as formulated does not have a single chance to succeed.--Ymblanter (talk) 10:21, 21 September 2017 (UTC)[reply]

Ok thanks. You should have been more clear that you are making the newbie and slimey mistake of Trying To Make a Big Deal And Even Quoting things that people obviously retracted before they were responded to. (I know how to step back when I make a mistake - I do understand that this is something rare around here). And thanks for consulting your crystal ball for me. Jytdog (talk) 10:42, 21 September 2017 (UTC)[reply]

I am not sure I understand whether this is meant as a personal attack or not, since English is not my mothertongue, but you are welcome to continue the rant, I am not going to respond to it.--Ymblanter (talk) 12:31, 21 September 2017 (UTC)[reply]

(e/c) And how would using the first line of the Wikipedia extract be playing content DJ for English Wikipedia ? That's my suggestion. I think it's totally unnecessary other than to keep the peace, but its what i'm suggesting none the less. —TheDJ (talk • contribs) 10:03, 21 September 2017 (UTC)[reply]

the WMF wants a single line of description it can use everywhere. I understand how that is very attractive. (i really do). It is however a huge deal for them to have made the decision to do so. Your solution is based on kissing en-WP ass and doesn't deal with the fundamental issues and in many ways is even more outside the norms of the movement than what the WMF did. Jytdog (talk) 10:13, 21 September 2017 (UTC)[reply]

I think in part, the WMF's role is to do things that communities and individual volunteers can't, like making apps, which have special app-specific features that need full-time product and engineering roles. I don't think that's an overreach. But I agree that that work needs to be done with the knowledge and the involvement of the communities, and that wasn't done well enough here. I'm hoping that having these conversations and working with folks here to come up with solutions is a step towards correcting that mistake. -- DannyH (WMF) (talk) 16:24, 21 September 2017 (UTC)[reply]

Making an app and making content decisions in an app are two different things, and you continue to obfuscate that. I am looking for the WMF to take these descriptions down and renounce the overstepping. I am not accepting what you are presenting as a fait accompli. Jytdog (talk) 17:55, 21 September 2017 (UTC)[reply]

I'm thinking about the discussion/argument above about descriptions of lists, albums, etc. On one hand, whatever the solution we end up with for short descriptions, there'll be plenty of time to figure out the standard format for rivers, museums, etc. But the conversation here is revealing, because it highlights a couple important questions that people have raised:

1.) Who's going to make the decisions about the standard format -- the Enwiki community, the Wikidata community, or both working together?

2.) Are Wikidata descriptions just for putting a heading on Enwiki articles, or are there other purposes they serve that would require a description to be different than what the Enwiki community would want?

Lists and disambiguation pages are an interesting example, because the Wikidata description isn't describing the subject of an article; it's describing the page. The description for the Gennady Golovkin article is "Kazakhstani boxer", which is useful both for a description on enwiki and any other use that someone might have for Wikidata information. But "Wikimedia list page" is not a very useful description for the enwiki app/search, and I can't think of any other use case where it would be helpful. It's kind of a description for description's sake, and seems like an example of something that enwiki could override with no real consequences.

If we can figure out a way for folks from Wikipedia and Wikidata to work together on making the descriptions appropriate for enwiki use, then that could benefit everyone. If not, then it seems to me like a template/magic word override on enwiki would make sense. There are a lot of useful descriptions on Wikidata, and I don't think it's practical for English Wikipedia to come up with five million descriptions from scratch. But using the helpful ones and overriding the unhelpful ones is a potential way forward. -- DannyH (WMF) (talk) 17:25, 21 September 2017 (UTC)[reply]

Wikidata descriptions, at the very least, serve to provide a proper choice on Wikidata itself - for example, when I am trying to fill in a property of an item, and I do not known the Q-number of the item I want to refer to, I just type an approximate name and get a bunch of options to choose from. Without descriptions, these options are not usable. With long descriptions, they will not be usable either. Every description, unless false, is better than nothing.--Ymblanter (talk) 17:34, 21 September 2017 (UTC)[reply]

Nobody has disputed the Wikidata editing community's use of the description field in Wikidata. That is their deal and none of en-WP editing community's business Jytdog (talk) 17:59, 21 September 2017 (UTC)[reply]

I was reacting to point 2 by DannyH (WMF) above.--Ymblanter (talk) 18:05, 21 September 2017 (UTC)[reply]

DannyH, you missed question zero as you have throughout this discussion. 0) Is it appropriate for the WMF to make content decisions? If (a big if) the answer is "yes", how should it interact with the content-producing communities? Jytdog (talk) 18:10, 21 September 2017 (UTC)[reply]

I think the way that the WMF should interact with the content-producting communities is the way that I'm doing it right now -- talking to people, and working collaboratively towards a solution that works for the Enwiki contributors, the Wikidata contributors and the app users. I agree with you that that hasn't always happened in the past. -- DannyH (WMF) (talk) 18:28, 21 September 2017 (UTC)[reply]

DannyH, by limiting the discussion to the English, you allow them to high jack the conversation. Thanks, GerardM (talk) 06:14, 22 September 2017 (UTC)[reply]

So that addressed question 0b, and leaves 0a unaddressed. Which is the primary issue here... which to restate the analogy, is "Putin in Crimea". I do not accept this fait accompli. Jytdog (talk) 22:02, 21 September 2017 (UTC)[reply]

DannyH (WMF), thanks. You missed #4 supplemented with a variation of #3: A Magic word to override the first sentence of the Wikipedia article.

To restate in this section: My !vote, and I believe the community preference, is #4. Adding a magic word override would be a nice plus. Alsee (talk) 03:47, 22 September 2017 (UTC)[reply]

@DannyH (WMF): you ask us two questions:

1.) Who's going to make the decisions about the standard format -- the Enwiki community, the Wikidata community, or both working together?

2.) Are Wikidata descriptions just for putting a heading on Enwiki articles, or are there other purposes they serve that would require a description to be different than what the Enwiki community would want?

Neither of these is in any way a relevant question if you go with the requested solution, and to pose these questions here and now gives the strong impression that you don't intend to do this or haven't actually read this page. There will be descriptions on enwiki, either directly (as text in a template probably) or at runtime (as text parsed from the first line and/or from the infobox). There will be descriptions on Wikidata. Wikidata is completely free to decide their standard format (including copying them from enwiki if they so desire), Enwiki is completely free to decide their standard format as well; the only thing we have to do at enwiki is discuss this with the WMF or whoever is behind the apps, search results, ... so that we get the best possible result, something which is workable in as many circumstances as possible. Your question 1 is thus a false trilemma: the correct answer, which you happen to omit, is "both working separately". Question 2 is probably even worse. If you want to know what Wikidata descriptions are for go to Wikidata and ask them. When you come here, ask enwiki what we want to use the descriptions for (or make suggestions of course). For us, "Wikimedia disambiguation page" is a useless description; I can imagine that for Wikidata, it is the perfect description. Fine, no problem, that's their choice, their right and I have no intention to make them change this in any way at all. That would be rude, unhelpful. Unless of course you continue to force enwiki to use these decriptions, whether we want to or not, which is the direction you seem to take again and again. If enwiki has to display these Wikidata descriptions, then they will have to be the text we want, and Wikidata be damned. This will only lead to a direct confrontation between the two communities and much ill will between the two (and even more towards the WMF probably). If that's what you want, just say so. If that's not what you want, then think harder about what you post here and whether it is in any way helpful. We've had enough fiasco's in the communication between enwiki and the WMF (which you, as the one responsible at the time Flow went down the drain here are certainly aware of), it would be best if we avoided this here and could continue with the more productive, open tone the WMF had adapted at the start from this discussion. Fram (talk) 07:20, 22 September 2017 (UTC)[reply]

Continued: you state "There are a lot of useful descriptions on Wikidata, and I don't think it's practical for English Wikipedia to come up with five million descriptions from scratch." For starters, as evidence above a few times, many, many enwiki articles don't have a description at Wikidata, hundreds of thousands have an unhelpful one, and many others have an unnecessary one (where the description on Wikidata is the same or similar to the parenthetical disambiguation on enwiki). These descriptions are useful on Wikidata, but not here. So even to get as good as the WIkidata descriptions are now, we wouldn't need to come up with 5 million descriptions, 1 million would probably be sufficient. And we don't need to come up with them from scratch, multiple possible solutions have been given, like initially copying the Wikidata descriptions (preferably in a smart way, e.g. dropping the ones for lists and disambiguation pages), using text from the first line, or using info from the infobox (if the infobox is an "infobox album", you can construct a description with "a "year of album" "type of album" by "artist or band" description fairly easily). Yes, it would take some bot coding and one massive bot run, but it isn't the first time something like this has been done. Fram (talk) 07:26, 22 September 2017 (UTC)[reply]

Fram, you and I are more on the same page than you think we are right now. I'm sorry that I wasn't clear about my intentions in that post; I'll clarify. This is going to be okay. :)

The reason why I'm asking questions in this discussion is that I want everyone here to have a chance to think and talk through some possible alternatives. When you say "Neither of these [questions] is in any way a relevant question if you go with the requested solution" -- I want to point out that that's the solution that you requested, which is a good one. Other people in this current thread are talking about other approaches or variations on that approach, and I want to leave some space for people to think them through, so that we can get to something closer to consensus. I'm not planning to spend the rest of our lives dragging this conversation out, but people are still actively discussing some different ideas.

When I asked those two questions, I was trying to summarize the discussion above my post, where people were talking about "Wikimedia list page", "Disambiguation page" etc. The answer to question #1 could be "the Enwiki community should make those decisions", and #2 could be "yeah, Wikidata has its own purposes, and it's just not appropriate to use that description field on Wikipedia." That's where that discussion was trending, but I wanted to leave some room for somebody to come up with a compelling argument for an alternative.

For the part about English Wikipedia coming up with five million descriptions: the solution you're championing involves importing all of the existing Wikidata descriptions. That means that you think there's value in starting with the work that's already been done on Wikidata, and then using some of it, changing some of it and building on that work. I agree with you that that would be more practical than the alternative that Jytdog is supporting, which is to scrub all the Wikidata descriptions completely, and start over from scratch.

So: if changing and building on top of the existing Wikidata descriptions is the direction we're going, there are two different ways that we could do it. One is to do the bulk import, and fork completely from Wikidata, which is your requested solution. Another is to use the template/magicword as an override, as we did with the "related pages" feature a while ago. That means good and useful descriptions added on Wikidata in the future could still appear, while English Wikipedia contributors could change the descriptions for the pages/topic areas where the Wikidata descriptions aren't good enough. There's pros and cons for both directions. I think it's reasonable to have some discussion about those alternatives. -- DannyH (WMF) (talk) 19:14, 22 September 2017 (UTC)[reply]

As the queries above showed, there are a fair proportion of the descriptions of Wikidata that could benefit from some systematic improvement, either before or in the process of any such import. But also the statements at Wikidata can often provide just the content needed to do just that. Jheald (talk) 19:34, 22 September 2017 (UTC)[reply]

From my perspective Options 2 and 3 would be helpful: keep drawing from Wikidata, but allow a local override. I don't mind making changes to the Wikidata descriptions, but others do. Let's allow a local override. I would also add an option 3a:

Option 3a: Allow a null override of Wikidata description for cases where the title itself is adequately descriptive. So, History of Iran and List of songs featured in Shrek just don't have a description below them.

Also, let's write a manual of style for this, and see whether our MoS and Wikidata's overlap or not.--Carwil (talk) 04:13, 24 September 2017 (UTC)[reply]

Still, History of Iran might be a notable book with its own article, and it would be useful for descriptions to discriminate it from the item on the history proper.--Ymblanter (talk) 07:42, 24 September 2017 (UTC)[reply]

Option 3 can effectively be converted into 4a, by a bot running at one edit per second for two months overriding wikidata on every article. However if that is the result community wants, it would be better to do so directly via 4a rather than via a bot-hack on top of 3. Alsee (talk) 08:31, 24 September 2017 (UTC)[reply]

@Ymblanter: That's what History of Iran (book) is for. There's no need for any description in that case as well.--DarwIn (talk) 12:06, 24 September 2017 (UTC)[reply]

Really? If you see History of Iran (book) you know immediately what it is about? I actually doubt this.--Ymblanter (talk) 12:30, 24 September 2017 (UTC)[reply]

@Ymblanter: Most certainly. And if the book actually is about the History of Iran, as it should be in 99.9% of those situations, there's no need for description at all. When you look at it, you wouldn't think it's a book about "Fishing in Portugal", would you?--DarwIn (talk) 12:56, 24 September 2017 (UTC)[reply]

I do not know about you of course but I happen to think that the name of the author of the book is essential information about the book.--Ymblanter (talk) 13:00, 24 September 2017 (UTC)[reply]

@Ymblanter: Sometimes it is, sometimes it isn't - many of the most important History books I know are the work of a large group of people under some coordination, and not of a single author. And History of Iran (book) is more than enough "to discriminate it from the item on the history proper", as you stated. Anyway, I understand your point. When/if a description is needed, add one, just don't use Wikidata as the prime source for it.--DarwIn (talk) 13:08, 24 September 2017 (UTC)[reply]

For users of the mobile app and the VisualEditor, the description does more than disambiguate, it also clarifies the exact topic being referenced. So, A History of Iran should not just say "book" in the description, it should say "2008 book by Michael Axworthy." And The History of the Siege of Lisbon should (and now does) say "1989 novel by José Saramago." Unlike disambiguation pages, the descriptions should aim to still work even as more items are added to Wikipedia/Wikidata.--Carwil (talk) 15:47, 24 September 2017 (UTC)[reply]

clarification

User:DannyH (WMF) I am trying to understand your role at WMF and in the WMF decision-making process about what to do with the descriptions. I checked the org chart and it appears that you are within "Community Tech" as a "Senior Product Manager". Would you please clarify - are you here writing as some individual employee of the WMF, or as someone with authority over decision-making on this issue? It is also not clear to me how your position relates to Jon's; would you please explain? Thx Jytdog (talk) 19:43, 22 September 2017 (UTC)[reply]

Sure, I can see how that's unclear. Jon is the product lead on the Reading team; I'm product lead on the Community Tech team. In my work on Community Tech, I've had a lot of experience over the last couple years working directly with community members on feature design, including some experiences where things got heated. I was talking with Jon and other people on his team about this discussion, and they asked if I could take lead on the feature-related communication, and help to find a solution. I'm not directly in charge of the product team that would work on making these changes, but we're all talking, and when we get closer to a conclusion on the community side, I can bring everybody together, and we can figure out requirements and timelines. -- DannyH (WMF) (talk) 00:40, 23 September 2017 (UTC)[reply]

OK, thanks for explaining. So, a) you are kind of "chief negotiator" for the WMF here, and b) you are looking to arrive at a concrete solution here. Please tell me if I have that wrong.

Three additional questions:

With regard to the goal of arriving at a conclusion here, is the intent that what is decided here is what will be implemented, or is WMF intending to take the conclusion to RfCs at the relevant communities?

Reviewing what you have said here, my sense is that the WMF is unwilling to take the description off of all en-wP content (the apps, visual editor, etc). Is that correct?

If that is correct, who is the person at WMF responsible for the decision to not take the description off?

Thanks again. Jytdog (talk) 01:05, 23 September 2017 (UTC)[reply]

Regarding Jytdog's last two additional questions: indeed, a lot of talking next to each other seems to be going on. Seems like, to use an analogy, WMF is trying to keep a car with a leaky tire driving (we'll figure out how to repair or replace the leaky tire while driving, in a sort of "think of the children" reasoning ("how would they get to school without the car?" or something in that vein)), while many of the Wikipedians in the discussion give clear signals the car would be better taken out of traffic first, and not allowed back in traffic before the leaky tire situation is remedied. The proverbial children may never reach school in the car driving around with a leaky tire. For the less technically inclined: continuing to drive around with a leaky tire is not so much an issue of loss of comfort, as the technical fact that it makes the tire heat up, and ultimately explode, due to friction causing heat and heat causing air to expand uncontrollably ("friction" and "hot air on the brink of explosion" fit well in the analogy of what I'm seeing on this page). So, indeed, who is the driver who can decide to take this car out of traffic before serious injury (read: some sort of loss of credibility for WMF projects, ultimately leading to less users) is caused? Figuring out repair modes is less prone to controversy during a period when the vehicle is temporarily not "live". --Francis Schonken (talk) 08:49, 24 September 2017 (UTC)[reply]

Well put.  — SMcCandlish ☺ ☏ ¢ ≽^ʌⱷ҅_ᴥⱷ^ʌ≼  13:01, 24 September 2017 (UTC)[reply]

Removing all of the Wikidata descriptions "immediately" would be a time-consuming engineering project that would take longer to implement than finishing this discussion and coming up with a better solution. It's really not a practical strategy. I understand why people aren't trusting the Foundation on this, since there was already an RfC. And I understand why people aren't trusting me, because why should you, and it's up to me to act trustworthy. I'm going to keep talking to people about a practical solution. I think the Crimean school bus is going to be okay for a minute. -- DannyH (WMF) (talk) 16:21, 24 September 2017 (UTC)[reply]

??? Who said anything about "removing all of the Wikidata descriptions"? I understand it is just about disabling an automatic coupling of "Wikipedia" and "Wikidata" in the two remaining apps that still make this coupling (and keep it off in all implementations where en.Wikipedia editors can't turn it off by removing a template etc). All the rest is "solutions", or not something for which WMF needs to intervene (whether or not Wikidata content is pulled via certain en.Wikipedia templates, is, as such, and putting it in as few words as possible, nothing for you to negotiate). Wikidata descriptions are of course useful within the Wikidata system, for instance when manually updating an item's properties. So, nobody asked to remove Wikidata descriptions from the Wikidata system.

@DannyH (WMF): seems the basic distinctions needed to negotiate in this discussion are lacking (e.g., distinction between "content", which nobody asks to remove, and "automatic coupling of Wikidata and Wikipedia", which the en.Wikipedia community has asked to switch off in all remaining apps that still do that *automatic* coupling). Maybe time to ask for a different negotiator, that is, one who can make these distinctions. --Francis Schonken (talk) 16:56, 24 September 2017 (UTC)[reply]

@Francis Schonken:: sorry, that's what I meant by "removing all of the Wikidata descriptions" -- taking out the automatic couplings from the apps. I didn't mean removing them from Wikidata. The concept of removing Wikidata descriptions from being displayed on the apps sounds simple, but is actually technically more complex than it sounds, because they're integrated in a lot of different places. It's not a grand project that would take months, but it would take more than the couple weeks that I hope it will take to figure out a better consensus solution to this problem. -- DannyH (WMF) (talk) 13:31, 25 September 2017 (UTC)[reply]

DannyH, that ("figure out a better consensus solution to this problem", combined with the previous sentence, sounds as if you have already decided that "taking out all the automatic couplings from the apps" is not the best (or even a good) solution. Why? That it may be hard for the developers (which may or may not be true, the WMF has too often used this excuse even when it was patently untrue) doesn't mean that it isn't a good solution or that we have to find a better one. You still seem to be pushing for a certain direction (pro-Wikidata), despite claiming that this is not what the WMF does. And of course, you (WMF) could take them out one by one, and keep us posted about the progress. Or you could actually present a concrete reply to some of the suggested better (!) solutions, like introducing a new, general template which (like Persondata) could be used across wiki-languages and could be read by the WMF where needed. As it stands, I see very little actual progress being made by the WMF, more stalling and pushing in a certain direction. Fram (talk) 13:41, 25 September 2017 (UTC)[reply]

• User:DannyH (WMF) thanks for posting here. Would you please answer the three follow up questions? Thanks! Jytdog (talk) 18:48, 24 September 2017 (UTC)[reply]
  • @Jytdog: just a friendly suggestion: maybe reformulate the second follow-up question a bit clearer, the part starting from "take the description off of all ..." which may have set Danny on a wrong foot regarding what exactly was being asked (if I understand correctly, Danny seemed to think that the request was to "remove all of the Wikidata descriptions", which is a question they answered with "no", but not really the question you asked, I suppose). --Francis Schonken (talk) 19:15, 24 September 2017 (UTC)[reply]

Jon listed the places where it is used well above in their section 4 which I will copy/paste here:

As you may know Wikidata appears in other places on Wikipedia. Wikidata descriptions appear not only below the title on mobile web in all languages except English and Russian, but in the following places:

• Apps under the title
• Visual editor (desktop and mobile)
• Portal search (www.wikipedia.org)
• Mobile web and mobile app search
• Related pages (bottom of mobile web and app pages)
• iOS widgets
• App maps feature
• Apps feed feature

The ones that are of the most concern are the first two, as both are explicit interference with en-WP content. I understand the descriptions have been taken down from the 4th in en-WP as a result of the March RfC. I have been uncomfortable with use of the Wikidata description field in the various navigation places -- as far I understand it, these uses arose in the course of the work of the Discovery team following the lines laid out by the Knowledge Engine debacle under Leila when the WMF completely lost its way and wanted to become The Great Aggregator of All Noncommercial Content... but I am not currently contesting that use for those purposes. I probably should, and may do later, but that is not the issue now. So to restate the three questions

1. Is the intent that what is "decided" here is what will be implemented, or is WMF intending to take the conclusion to RfCs at the relevant communities?
2. Would you please confirm that WMF is unwilling to remove the Wikidata descriptions from Apps under the title and Visual editor (desktop and mobile) for en-WP content?
3. If that is correct, who is the person at WMF responsible for the decision to not take the description off these places?

-- Jytdog (talk) 19:33, 24 September 2017 (UTC) (removing VE as this is a navigation thing Jytdog (talk) 18:05, 25 September 2017 (UTC))[reply]

"as far I understand it, these uses arose in the course of the work of the Discovery team following the lines laid out by the Knowledge Engine" eh.. that seems extremely unlikely to me. Other than the portal work, all those usages originate from different teams. I think some even predate that entire team... —TheDJ (talk • contribs) 12:38, 25 September 2017 (UTC)[reply]

@Jytdog: P.S... what makes VE a content usage in your view? To me the usage by the link inspector seems more similar to the other usages, than to the subtitle usage... —TheDJ (talk • contribs) 15:00, 25 September 2017 (UTC)[reply]

Perhaps I am misunderstanding: i don't use VE and didn't actually go check. By listing "VE (desktop and mobile)" this way, I believed that the description field is what you see at the top of an en-WP article when you are using VE on web or mobile. Is something else meant? Jytdog (talk) 15:10, 25 September 2017 (UTC)[reply]

That's absolutely not how the VisualEditor uses these descriptions. It offers a drop-down menu for new links—see image on the right—which is (as I said above) exceptionally useful for people editing a page. When the link is chosen, you don't see the description, nor the does the VisualEditor help you see or edit the Wikidata description of the page you are currently editing.--Carwil (talk) 17:28, 25 September 2017 (UTC)[reply]

I see, so it is a navigation thing. As I noted above, this is... problematic but not what I am focused on now. Thanks for explaining. Jytdog (talk) 18:05, 25 September 2017 (UTC)[reply]

Hi everyone: Based on the conversations that we've had here, I've made recommendations to the Reading team and to Product as a whole. It's going to take people a day or two to talk about it, just because there's a bunch of people to check in with, in different time zones. I don't want to say much about what the recommendations are just for the next day or so, because I don't want to end up making promises that I couldn't keep. I'll definitely get back to everyone here by end of the day Wednesday (in SF) -- if I don't have anything clear to say then, I'll at least check in and explain what's going on in more detail.

So, to answer Jytdog's questions: #1. My intention is that decisions here will be implemented. There was already an RfC in March, and based on the discussions here, I don't think another RfC would be much different. But, like I said, there needs to be conversations with a bunch of people. #2. Right now, while this conversation is going on, (ie for the next couple days, and maybe couple weeks if things drag out) the WMF does not want to pull all of the Wikidata descriptions out of the apps before we've all discussed and agreed on a solution. That's what I've understood from Jytdog's posts over the last week -- that the Wikidata descriptions should be pulled from the apps immediately (as in, today) before we discuss or decide on a solution that gives Enwiki editors more control over the descriptions. If the question is "Would you please confirm that WMF is unwilling to ever remove the Wikidata descriptions," then the answer is no, we are not permanently unwilling to to ever remove them or give Enwiki editors more control. Sorry, I'm trying to be super extra clear, but there are a couple double negatives involved. :) #3. The decision to not remove the Wikidata descriptions this week is shared among pretty much everyone at the WMF, because we want a couple more days to talk about it and figure it out. The decision on whether to remove the description or give Enwiki editors more control has not been made; we'll be talking about it, and I'll definitely check back by Wednesday. Sorry I can't be totally definitive right now. I know that it's frustrating that people on this page have been talking about this since March (or longer) and you're not getting a definitive answer right now. -- DannyH (WMF) (talk) 13:51, 25 September 2017 (UTC)[reply]

Thanks for your replies. The notion that the WMF is "considering" giving the en-WP community "more control" over en-WP content is completely upside down. I will until your Wednesday reply before deciding my next steps. Jytdog (talk) 18:19, 25 September 2017 (UTC)[reply]

DannyH (WMF) as long a reasonable progress is being made, I accept that it would be more efficient to leave the current system in place while building an agreed-upon-solution. On the other hand, I'm feeling a bit wary/paranoid about the vague way you phrased "give Enwiki editors more control". If that means continuing to use wikidata descriptions and providing 'more control' by increasing wikidata-integration - then I am willing to help draft an RFC on the proposed enhancements. However I don't support it and I suspect the proposal will fail. Alsee (talk) 18:45, 25 September 2017 (UTC)[reply]

Alsee, if in their Wednesday response the WMF doesn't say it is taking the descriptions down from the apps and that it will seek consensus from the relevant editing communities before re-instating some new version, I am thinking that I will re-instate an RfC along the lines of the one you asked me to take down. In my view this is somewhat of a movement constitutional crisis. The folks from WMF don't understand the fundamental deal here (volunteers generate content following their project policies and guidelines; the WMF publishes it) nor do they understand the importance of consensus nor even why that process is so fundamental in this environment. (It is telling that they were going to run with whatever the few people talking here had suggested.)

I realize that the descriptions are just a few words but the principles here are important to me, and I think they are important to most other people in en-WP as well. The world should not remain upside down. Jytdog (talk) 19:22, 25 September 2017 (UTC)[reply]

Isn't stripping functionality for 5.6 million Android users, unknown millions of iOS users ([2]) on the Mobile App, and thousands of VisualEditor users because of "a constitutional crisis" between active editors and WMF exactly what WP:POINT is meant to try and prevent? Why are we not talking about this in terms of the experience of readers: What helps them navigate? What helps new editors write (and link)? What will cause the least astonishment: fixing a problem iteratively or ripping out functionality and then putting it back in?--Carwil (talk) 17:26, 27 September 2017 (UTC)[reply]

Putin: But look, I really improved the highways in Crimea! Isn't that great?! I mean, look at this metric I created -- all the numbers are great!! I am improving Crimea while all you people just stand around and wring your hands over trivia and criticize me. Stop getting in my way and help me improve Crimea!"

Rest of the world: "....." Jytdog (talk) 17:30, 27 September 2017 (UTC)[reply]

When the WMF builds a highway—I mean an app—it appears simultaneously in every country—I mean language. The "rip it out" request requires forking the Wikipedia app to be less functional in English than any other language. Again, we and the WMF collaborating and interdependent, not sovereign and independent. Anyhow, you're just reinforcing that this is about making a point, not improving the encyclopedia.--Carwil (talk) 17:39, 27 September 2017 (UTC)[reply]

Putin: "The territorial imperatives of Russia are not a matter for you tiny people to debate. Now who will help with the highways in Crimea!? Where is my 'attaboy!' for this great work I did!?"

Rest of world: "................." Jytdog (talk) 19:45, 27 September 2017 (UTC)[reply]

Hi everyone: I said on Monday that I would let you know what's going on today. The product team is still talking things over, so I don't have anything substantial that I can talk about yet.

There are a bunch of people from several different teams who are actively talking about this question right now, and what our response should be. People are coming at this from different perspectives, and as you know, sometimes it takes a while to reach consensus.

We're having a meeting tomorrow morning to talk things over. I don't know if we'll have a clear answer yet, but I'll definitely come back to this page tomorrow and let you know how it's going. This issue is important, and we're taking it seriously; we want to make sure that we're on the same page, when we come back here and talk with you all again. Sorry for the delay, I'll write more tomorrow. Thanks. -- DannyH (WMF) (talk) 00:38, 28 September 2017 (UTC)[reply]

Proposal from WMF

Hi everyone: As I mentioned yesterday, the product teams had a discussion this morning about the Wikidata descriptions. I'll tell you how we're thinking about the problem, and what we think would be a good direction for the feature. We're seeing this as a continuing discussion, so we want to know what you think.

Short descriptions are useful in search, and they're essential to features that are used very heavily on the apps, including the list of Top Read articles. Removing the descriptions completely would be damaging to the readers' educational experience, with no benefit to anyone.

But there is a problem with editorial control and dispute resolution, which we didn't pay enough attention to. When there's a content dispute on English Wikipedia, there's a structure for how to resolve it, with a way to escalate the problem if the editors can't resolve it amicably. We don't have a structure like that for cross-project work, so if there's a difference of opinion between Wikipedia editors and Wikidata editors, it's difficult to resolve. The existing feature gave that editorial control to Wikidata, because they had a structure for defining a short description. But through this discussion, we're understanding that if the page we're showing is from English Wikipedia, then it's appropriate for English Wikipedia to be the center for editorial control. So we're thinking of the proposed approach as a cross-project default with a local override, which gives the local community the ability to resolve any differences. This is similar to the way that we use images from Commons, or interwiki links from Wikidata.

We're proposing a new magic word -- {{#D:description}}, or something similar. That local description would be the one that's used. If there's no magic word or the parameter is left blank, then it defaults to the Wikidata description.

We think that it wouldn't make sense for English Wikipedia to start from scratch and create millions of descriptions, so the local override would allow people to use the existing Wikidata material that's good, and override it when there's a reason to fork from the Wikidata version. We'd like to encourage people to continue partnering with the Wikidata community as a source of new and updated descriptions, but we also acknowledge that once this feature is set up, it's up to the English Wikipedia community to decide how to use it.

If this is where we're going, then we'll also talk about how editing descriptions in the app should work. Right now, on all languages but English, users on the Android app can edit the Wikidata descriptions. We could work on supporting editing the local exceptions instead, but we'll need to figure out how the user interface would work. We're still talking about it, and we're interested in what other folks think.

Now, if descriptions are still coming from Wikidata as the default, then people will want to be able to monitor changes from Wikidata. Right now, you can see Wikidata edits in your watchlist if you uncheck "hide Wikidata" or enable the Wikidata edits filter, but that includes all edits to the item, not just the English description. The Wikidata team is currently working on showing only the locally relevant Wikidata properties -- a test version of that feature is currently live on Greek Wikipedia -- but it's too early to say when that feature would be available for English.

Like I said, we'd like to continue this discussion with folks here, to find out what you think and see if we can get closer to a consensus on this feature. While this discussion touches on larger questions -- like governance on cross-project features, inter-project dispute resolution, and Wikidata integration in Wikipedia -- I think that this discussion isn't the right place to resolve those. I'd like to keep this conversation focused on the practical things that we can do to restore English Wikipedia's editorial control for the short descriptions. If you think this requires bigger conversations, we're open to having those discussions, but I want to work on tangible solutions here. So that's where we are right now. What do you think? -- DannyH (WMF) (talk) 23:31, 28 September 2017 (UTC)[reply]

This is not like how images from the Commons are used. Use of images from the Commons in en-Wikipedia is a) driven by en-WP editors who add images to en-WP voluntarily one by one, and b) is governed by en-WP's WP:Image use policy.

When you say "We'd like to encourage people to continue partnering with the Wikidata community", I hear "extortion".

The Wikidata field will show, and if it violates en-WP policy, then it is up to en-WP editors to take action to override it, and everybody has to learn some weird template to do that. I do not find it all reasonable that WMF dumps a bunch of unsourced content into en-WP and then just leaves that for en-WP volunteers to fix (and that you would "give" us tools to fix it, is even crazier.) I have said this before but the arrogance and ignorance of this, is just beyond my understanding. And I really believe you have no idea at all how batshit crazy what you are saying is or how blatantly the WMF is displaying power and disdain.

Putin: "So I get it that that when we changed all the highway signs in Crimea to Russian that has messed with people and we have done some mistranslations maybe. You all can fix those signs where ever you find them - I will give you all some paint brushes. OK, go get moving. Aren't these highways great?!"

Rest of world "...................... "

Please listen to the metaphor. This is what you sound like.

You never got consensus to systematically add content to Wikipedia. You just moved on in. Jytdog (talk) 03:05, 29 September 2017 (UTC)[reply]

• It was a (very) bad idea to give Wikidata editors control over Wikipedia content. Giving Wikipedia editors a gizmo with which they can control Wikidata content is a stupidity of the same order, and doesn't even begin to address the first issue, which was, summarized: one WikiMedia project should not decide on the content of another WikiMedia project. Wikipedia editors do not control the content of Commons (which seems to be a misunderstanding in Danny's explanation above), Commons editors do not control what Wikipedia editors decide to retrieve (or not) from Commons for use in Wikipedia. --Francis Schonken (talk) 05:56, 29 September 2017 (UTC)[reply]

Another way to think about this, is that WMF has essentially run a bot to systematically add content to en-WP articles. en-WP has a bot policy: WP:Bot policy. Did WMF go through the process described in that policy to run the software that made these systematic changes to en-WP content in the apps? Jytdog (talk) 06:00, 29 September 2017 (UTC)[reply]

"So we're thinking of the proposed approach as a cross-project default with a local override, which gives the local community the ability to resolve any differences. This is similar to the way that we use images from Commons, or interwiki links from Wikidata." As has been siad countless times now, this is not how we use images from commons. With Commons, we decide which image, if any to use, no images are sent by default ever (I hope, or has the WMF done this as well somewhere?). Interwiki links is exactly what Wikidata has been set up for, and where enwiki and the others voluntarily relinquished the lead. But this is not content shown in articles and seen as the first thing by all mobile or app users.

"We're proposing a new magic word -- {{#D:description}}, or something similar. That local description would be the one that's used. If there's no magic word or the parameter is left blank, then it defaults to the Wikidata description." No, no, no! The default is enwiki. There may be an override where we can explicitly say "use the Wikidata description instead", but in many cases we will want no description at all, or certainly prefer "no description" over "the Wikidata description". The examples of lists and disambiguation pages have been given, or something like Spirit (Depeche Mode album), which doesn't need a description here, and certainly not the utterly redundant "Depeche Mode album" used as description on Wikidata (where this description makes 100% sense). At the very, very least, either "no magic word" or "magic word without description" should result in "no description" instead of the Wikidata one. My preference would be "no magic word = no description".

"We think that it wouldn't make sense for English Wikipedia to start from scratch and create millions of descriptions, [...]" I doubt that Wikidata hsa "millions of descriptions" we could use, if you discount all lists and disambiguation pages, and most or all pages with a disambiguator on enwiki (which already serves as a description in many cases), and take into account all pages on Wikidata without a description anyway. Bot adding the magic word to all pages which don't meet the above exceptions shouldn't be too hard anyway. Using a smarter bot instead which creates descriptions from either enwiki first lines or enwiki infoboxes may be even better of course.

"We'd like to encourage people to continue partnering with the Wikidata community as a source of new and updated descriptions [...]" Why? It is not the role of the WMF to encourage or discourage such things, and the descriptions on Wikidata very often serve a different purpose than the ones here. It doesn't make sense to look at a much smaller, multilingual community to be the source of descriptions on enwiki, when the descriptions on that community often serve a different need in the first place.

"We could work on supporting editing the local exceptions instead, but we'll need to figure out how the user interface would work. We're still talking about it, and we're interested in what other folks think." Well, it would be rather ridiculous to show people the enwiki description, and then have an "edit" functionality that sends them to Wikidata, where any change they make won't affect the end result anyway. Which is a god reason to drop the "use Wikidata decriptions under circumstances X, Y and Z anyway" approach and go for a full and only enwiki approach instead. Fram (talk) 06:45, 29 September 2017 (UTC)[reply]

Note: I've added this discussion and proposal to WP:CENT[3], feel free to amend the note if it is unclear or biased. Fram (talk) 06:56, 29 September 2017 (UTC)[reply]

Note 2: now also added to WP:VPPR, WP:AN and WP:BON. Please drop a note elsewhere if necessary. Fram (talk) 07:08, 29 September 2017 (UTC)[reply]

Bottom line: WMF thinks that short descriptions of some kind are essential, and doesn't believe the airy assertions above that 5 million perfect pink unicorns can be created here overnight. Instead, it suggests going with the nags we already have, even if some of their faces need a wash; and trusting in the wiki way of improving these where they fall short, by opening them to the community and letting the community improve them. (It's also a lot easier to improve them and systematise them at scale on WD than it would be here). In some cases they may not be great, but in most cases the current descriptions or auto-descriptions are better than nothing and at worst mostly harmless. The proposed over-ride mechanism lets editors improve descriptions from in here if they want to, and start hand-crafting bespoke descriptions for WP where we can be bothered -- eg in the first instance perhaps for the pages that are highest rated or most read or considered to be at a particular high risk. And probably a mechanism will evolve to offer these locally optimised descriptions back to Wikidata.

User:Jytdog might like to reflect on a point that earlier versions of one of the policy pages, perhaps WP:NOT, were quite brutally blunt about: WP is not a democracy. Ultimately it's WMF that owns the servers, and they can and will do what they want. If you don't like that, WP is not WP:COMPULSORY. Jheald (talk) 08:05, 29 September 2017 (UTC)[reply]

To be honest, I do not see any problem. There is an RfC that descriptions should be not taken from Wikidata automatically. There is a common wisdom that they can be taken from Wikidata, but many people, some of whom are especially vocal, believe this is not appropriate. I think the solution is obvious: (i) make a mechanism which would only show a Wikidata description on the English Wikipedia if the description was created from the English Wikipedia (similarly to interwiki links) or validated from the English Wikipedia in a similar way; (ii) instantly disable and do not show Wikidata descriptions unless validated. This is the only way to let people to start caring, and it should not affect any other projects.--Ymblanter (talk) 08:11, 29 September 2017 (UTC)[reply]

We can do more than nothing. And I don't think WMF actually wants to pull this kind of power play. I really hope not. Jytdog (talk) 08:28, 29 September 2017 (UTC)[reply]

This is also how I personally see it: since descriptions are tightly bound to article and language (this en-Wikipedia in this case), the solution which would make the most sense to me would be not for WikiData entries to be overridable by a special/magic variable, but for it to originate from en-Wikipedia directly, considering that it's only a short text which could be as part of a template. A bot could initially import existing descriptions into en-Wikipedia articles. However, I understand that from an implementation's standpoint the existing code depending on WikiData and perhaps the mobile applications might need modifications if this was done. An alternative would be to support the proposed override magic variable and to use a bot to override all existing uses, such that the code requires less changes while technically obtaining a similar result... On the other hand, while new articles are created, more and more articles could end up with descriptions from WikiData in the long term (unless a regular robot run systematically moved to en-Wikipedia any WikiData-originating description). —PaleoNeonate – 08:32, 29 September 2017 (UTC)[reply]

I am sure 95% descriptions can be generated by bots and be acceptable on both English Wikipedia and Wikidata. I think the real question is what to do with the existing description. I proposed to not show them on the English Wikipedia unless explicitly validated (possibly by bot)--Ymblanter (talk) 08:42, 29 September 2017 (UTC)[reply]

• In the event any sort of 'magic word' was implemented, it would be trivial to create a bot to run through the ENWP articles making sure none get descriptions from wikidata. It might be a large number of articles (as Fram has pointed out it might not be as large as expected) but that would still be a matter of time to work through, in the weeks rather than months. If the end result is WMF using valuable engineering time in order to implement something we can then use to actively prevent any Wikidata descriptions being shown, it would be time better spent just disabling the descriptions from wikidata from your end. Its a waste of time and money implementing something that will only be used to bypass a *previous* waste of time and money. Only in death does duty end (talk) 09:26, 29 September 2017 (UTC)[reply]

  There's a lot coming up here, but I want to respond to one thing in particular. As I said yesterday, we want to give English Wikipedia control of the descriptions. When we're figuring out possible solutions with you all, the thing that makes it hard for us is the idea of blanking all of the current descriptions.

  I agree with Fram that the Wikidata descriptions that describe the page rather than a subject are useless -- "Wikimedia list page," "Wikimedia disambiguation page," and any other examples of that kind of thing. Those should be removed, and that's something we can do. But there are lots of helpful existing descriptions that make using the app a better learning experience, and the idea of going from x million descriptions to zero, even for a short time, is hard for us to agree to. That's what made us lean towards local exceptions, rather than automatically pulling all descriptions from Wikipedia. I think that we could come to a productive, workable solution that gives control back to English Wikipedia where it belongs, if that solution doesn't involve a period when there are no descriptions displayed at all. What do you think? -- DannyH (WMF) (talk) 22:52, 29 September 2017 (UTC)[reply]

  Why is it such a hard thing to agree to? It would indicate that WMF is willing to accept Wikipedia control over Wikipedia content. Retaining the descriptions shows a reluctance to leave Wikipedia content/appearance in the control of Wikipedia, and sends us a message that (somebody at) WMF is dead set on continued interference and on winning this battle, which should not even be a battle. This reluctance is likely to escalate the problem. · · · Peter (Southwood) ^(talk): 06:09, 30 September 2017 (UTC)[reply]

• "If there's no magic word or the parameter is left blank, then it defaults to the Wikidata description." What is the justification for this proposed behaviour? I can understand, if not necessarily support, wanting to continue using the WD descriptions as default, and using the magic word to "opt out" for a localised description instead, but articles are not created containing magic words by default: editors place them there deliberately. And it follows that if a blank {{#D:}} tag is in an article, it is because editors feel the description should be left blank. The null parameter should be respected as a null description. Please don't make use pass a single non-breaking space to all the pages we feel are better left without descriptions. Snuge purveyor (talk) 08:52, 30 September 2017 (UTC)[reply]
• I have not seen objections which address the key point mentioned just under "Proposal from WMF", namely that descriptions are very useful as they allow the software to give better results when people search for topics, and they allow showing a brief outline of what an article covers when listing Top Read articles. Those objectives should be supported. Johnuniq (talk) 10:13, 30 September 2017 (UTC)[reply]
  • @Johnuniq: Personally, I find descriptions very useful, and I applaud that idea to make such thing appear at certain places, such as the search list, or the mobile app. In Wikipedia I've the article preview gadget activated, which does a similar function (and sometimes in a better way), and I recognize the great value of those short descriptions. But they should be entirely placed on the Wikipedia side, and under Wikipedia control. That's where content belongs to, that's where it can be properly curated, maintained, updated. Wikidata is useful for interwikis and possibly other things, but should never had been brought into this process. They should be excluded from it as soon as possible. Wikidata here it's only messing up things, with no gain at all. Generate short descriptions on the fly from the leading sentence/infobox from Wikipedia articles, seems to me by far the best option, if it's possible to implement. If Wikidata wants those descriptions as well, they can copy them afterwards, and do whatever they would like to with them.-- Darwin ^Ahoy! 12:31, 30 September 2017 (UTC)[reply]
• I don't think this proposal is going to work, because it is a compromise. More important, I think it puts WMF in the awkward position of having to defend this compromise, while the community itself is still deeply divided. This opens up the foundation to a neverending amount of wikilaywering that they have to deal with. This community clearly has no interest in evolving any of this, has no consistent message for WMF and no plan. Just return it back to status-quo pre-2015, so that we can move past the drama. I'm not saying we should not have this functionality on English Wikipedia, but by now I've arrived at the point, where as WMF I wouldn't work on this until the community has made a design for it and gets sizable portion of the community to sign off on that design. It's time to let the editors do the software development, they clearly have a desire to work on this. This has gone on long enough, the foundation has more important stuff to work on. —TheDJ (talk • contribs) 18:00, 30 September 2017 (UTC)[reply]
• DannyH (WMF), I think there is a lot of understanding for your concern: thing that makes it hard for us is the idea of blanking all of the current descriptions. Maybe it will help if I explain why that option keeps coming up. Many on the community side find it hard that the WMF keeps turning the discussion to blanking descriptions. I believe the effective community consensus would be for using the lead sentence, preferably with the override option added. That would effectively give descriptions for all articles. Major win. The reason 'blanking' comes up is because we can't build the desired solution from our end, and we do not presume to be able to order you to build it. When the WMF drops that option from the discussion, we naturally turn to the options that you leave available to us. When you narrow the discussion to "The WMF won't discuss anything except Wikidata descriptions", people who think this shouldn't be Wikidata at all are cornered into the "blank it" option. I don't know if the WMF is willing to discuss non-Wikidata solutions, but hopefully this helps you and others at the WMF understand why the discussion is going the way it is.
  Remember when I said I was wary and paranoid about what response you would bring back from the WMF side? I am disappointed, but completely unsurprised that you silently excluded the non-Wikidata option from discussion. You haven't explicitly declared it "out of scope for discussion", but your response here has implicitly done so. You are effectively asserting that "blank it" is the only alternative remaining on the table. Alsee (talk) 04:30, 1 October 2017 (UTC)[reply]

  Alsee, thanks for saying that. Actually, using the first sentence isn't out of the question -- you're right, I should have talked more about that option. I think you're right that blanking has come up because of WMF stonewalling, and then we respond to that by getting more nervous about community control, and the cycle continues. I should be thinking outside that cycle. We're going to have more internal conversations about solutions today, and I'll make sure that deriving descriptions from the first sentence is part of the discussion. I'll report back later today on what we're thinking. Thanks. -- DannyH (WMF) (talk) 16:29, 2 October 2017 (UTC)[reply]

Okay, we talked about it some more today. If folks here don't feel like the Wikidata-hosted descriptions with local override gives enough control, we're open to another idea which has been proposed on this page, which is copying the existing descriptions from Wikidata to English WP using the magic word, at which point everything is under English WP control, and people can decide on appropriate standards as people usually do on wikis. I'm interested in hearing people's thoughts on that alternative.

We talked about pulling descriptions from the first sentence, rather than copying the existing description from Wikidata, and it doesn't sound to me like it would be an improvement. It's possible to write some code that pulls out the subject, the words "is a", the pronunciation, etymological data, birth and death dates, etc, and end up with a short description. But those descriptions would be flawed, in the way that algorithmically-generated text always is, and in the past, English Wikipedia editors have objected to doing things algorithmically that could be done better by hand. The majority of existing descriptions are already the same thing that what we'd want to generate anyway.

I'm going to do a quick comparison here, using the 33 pages listed in today's Top Read module. I'll post screenshots so you can see where these come from, and then compare the existing description and what we'd want the algorithm to pull from the first sentence. Here's the comparison:

Here's the list:

Now, I would want to improve some of those existing descriptions, but on the whole, I think the existing ones are better than the generated ones would be. The generated descriptions are often longer, and a human is better at deciding which words are important and which aren't.

So I don't think the existing Wikidata descriptions are toxic; they don't need to be killed with fire. They're mostly fine. I agree with folks here who say that English Wikipedia should have editorial control over the descriptions, and I want to figure out a process that gets us there without having a period where users don't see any descriptions at all. What do you think? -- DannyH (WMF) (talk) 02:07, 3 October 2017 (UTC)[reply]

Looking at the most viewed pages obviously gives a biased result about how good the descriptions are, as they are more often viewed and a specific type of articles (not e.g. the lists and disambiguations). In any case, I agree that it would be better not to have a period without descriptions, if this can be achieved in a reasonable time period. Not having descriptions for a while isn't too terrible either, we did well without them for a very long time anyway, but if we can achieve a smooth and relatively fast transition then so much the better. Basically, 1. the WMF has to tell us which template or magic word they would support (this should come from WMF an dnot from us, as perhaps other wikis may want to follow the same path), 2. enwiki has to decide how to implement and fill that template/magic word, 3. a bot must do the actual implementation, and 4. the WMF needs to change (application by application presumably, no need for a big bang) all instances where this description is now used / shown. Fram (talk) 08:09, 3 October 2017 (UTC)[reply]

DannyH (WMF), I'd like to confirm my understanding. Is it accurate to say the current proposal is (1) a bot to add {{#keyword:copy of wikidata description}} to articles then (2) turn off Wikidata descriptions? An article without #keyword would have a blank description?

Assuming I understood it correctly, I think this should be agreeable. I think we would want #keyword inside a template and have the bot insert the template instead. That would give us more flexibility. For example the template might apply a category or preview message if the description is too long. It also avoids putting a #parserfunction: directly in articles. It's easier for people (especially new users) if everything is {{xxxxxx|value}} rather than randomly mixing in {{#xxxxxx:value}}.

Assuming others on this page are good with this option, I'd like post the proposal as an RFC. I think it will be an easy consensus and there are benefits to the RFC itself. It ensures we catch any other concerns, it effectively publicizes the plan, and it provides something to point to if anyone complains later. I also think it's helpful if we can experience more RFC's as positive tools and success stories between WMF&community. Alsee (talk) 18:19, 3 October 2017 (UTC)[reply]

Hi, FYI: We're having more internal conversations about these ideas with the product teams. It's a big group of people, and coming to consensus is hard. I'll post tomorrow with an update about where we are. Sorry to go dark for a minute. -- DannyH (WMF) (talk) 22:42, 4 October 2017 (UTC)[reply]

Hi everyone, another update: We've been talking a lot about this issue internally, and we need to have a meeting with the product teams, so we can come to consensus on what our approach should be. That meeting is scheduled for Wednesday. I know, that's six days from now, but there's a three-day weekend in the US, and this is the earliest when we could get everybody together. I'll have a substantive update for everybody after that meeting on Wednesday. Thanks for being patient with us while we talk about it. We're taking it seriously, and we don't want to short-change the discussions. -- DannyH (WMF) (talk) 00:33, 6 October 2017 (UTC)[reply]

Hi everyone, we were hoping to get some plans together today for addressing the moderation concerns around the Wikidata descriptions. But there are some current database issues related to listing Wikidata changes in watchlists and recent changes, which are being discussed on this Phabricator ticket. We need to see what happens there, before we can make plans around the descriptions. I'm sorry about that; we'll see what we can do. -- DannyH (WMF) (talk) 00:43, 12 October 2017 (UTC)[reply]

Thank you, DannyH (WMF) and the rest of the WMF team, for recognizing the seriousness of that situation. Risker (talk) 05:46, 12 October 2017 (UTC)[reply]

At best, we get back to the situation where Wikidata can be used without breaking watchlists and recent changes. At worst, Wikidata usage will be heavily restricted. How does any of this have an impact on a solution to get rid of the Wikidata descriptions completely on enwiki? It seems as if either (and most likely), it won't have any impact on the other problems, or it will help alleviate the server load. If the problem is that the same people work on both issues, then of course such a serious problem takes precedence for now: but if the people working on it are not the same, then I don't see how it is a reason to postpone a solution here even further. Fram (talk) 12:45, 12 October 2017 (UTC)[reply]

DannyH (WMF), When can we expect an update from WMF? I notice that my watchlist is timing out on average 3 times on an incomplete script before it renders completely, and there are large numbers of Wikidata edits listed which are in no obvious way related to Wikipedia and do not appear to display on desktop in any way.

Re-pinging DannyH (WMF), maybe you did not get a notification. · · · Peter (Southwood) ^(talk): 12:26, 19 October 2017 (UTC)[reply]

I have been doing some editing on Wikidata relating to my work on Wikipedia and Wikivoyage, and so far I have seen nothing to suggest that the average Wikidata item has any place being imported to Wikipedia. · · · Peter (Southwood) ^(talk): 06:07, 17 October 2017 (UTC)[reply]

Ah, I also got script errors on my (long- watchlist), but I thought it was my machine. No idea if it is Wikidata-related or not though, the Wikidata-on-watchlist problem is supposed to be under control (FWIW). Fram (talk) 07:25, 17 October 2017 (UTC)[reply]

My watchlist is 1,641 pages - moderately long? I get much the same result on desktop with Firefox and laptop with Chrome, takes a bit over 30 seconds to complete. · · · Peter (Southwood) ^(talk): 07:43, 17 October 2017 (UTC)[reply]

Mine is 10,029 pages, I do not get any errors (Firefox on laptop).--Ymblanter (talk) 07:46, 17 October 2017 (UTC)[reply]

Hi Peter, thanks for the re-ping. According to the Phabricator tickets, the watchlist issues on most wikis are fixed (phab:T171027), but the database admin is testing and cleaning up some smaller wikis (phab:T178290). I'm not sure what the decisions are going to be yet, but I'm hoping to learn more tomorrow. -- DannyH (WMF) (talk) 18:40, 19 October 2017 (UTC)[reply]

@DannyH (WMF): can we please get some timeline and indication of what actions are being taken? This starts to look suspiciously like a very long exercise of "everyone is busy, please wait a bit longer" but without the typical Vivaldi music to make the waiting seem less annoying. It's especially bizarre that in other discussions, people tell us that the watchlist / recent changes issue isn't that important, already solved, not affecting enwiki (which already has on average the most problems with the delay between the changes on Wikidata and them appearing here in recent changes); while here the same problems are the reason that an answer (which was already long in the making) isn't forthcoming now. Fram (talk) 13:47, 23 October 2017 (UTC)[reply]

Hi Fram and all, sorry about the delay. The Wikidata team is now working on the problem of making the watchlist entries more specific, and they're making progress on specifically tracking the description as a single item -- the ticket is phab:T106287, if anyone's interested in that. Now that that crisis is being handled, we're going to meet this week to talk about what we're going to do about the descriptions, and (more importantly) who's actually going to be responsible for doing it. I'll be able to post an update by Friday, when I'll have more information. Thanks for being patient over the last couple weeks. -- DannyH (WMF) (talk) 00:00, 25 October 2017 (UTC)[reply]

This may help speed up watchlist rendering, but will not solve the problem that the descriptions of Wikipedia articles are not curated on Wikipedia. · · · Peter (Southwood) ^(talk): 20:58, 27 October 2017 (UTC)[reply]

Magic word

Hi everyone, thanks for being patient while we figured out what to do with the Wikidata descriptions.

We think that using a magic word on Wikipedia articles to override the Wikidata description is a good approach here. The description on Wikidata would be used by default, but when the description isn't good enough, then editors on Wikipedia can override it. This gives editorial control to Wikipedia editors, while still using the work that people have added on Wikidata.

Using both the Wikidata default and the Wikpedia override makes it more likely that articles will have descriptions, especially new articles. When a new item is created on Wikidata, there's a strong call to action in the UI to write a description -- it's right at the top of the page. There isn't a direct call to action to encourage adding descriptions on new Wikipedia articles, so people may be less likely to add a description for every new page.

The magic word won't work if it's blanked, because the descriptions are useful and we don't want to lose them. But various people in this discussion have pointed out that some kinds of pages don't need descriptions -- list pages, disambiguation pages, category pages, the main page, and anywhere else where the description is describing the page, rather than a subject. "Wikimedia category page" and "Wikimedia list page" aren't helpful. As part of this project, we'll hide those from display.

Also, the Wikidata team is currently working on making it easier for Wikipedia editors to monitor the Wikidata descriptions, by separating out the English descriptions from other changes. (The tracking ticket is here if you like tracking tickets.) The work that we do on the magic word will be parallel to that work, they don't depend on each other.

So that's our current plan. We still need to talk to some developers on the team about the technical feasibility of this change, and how it slots into the existing product roadmap. Right now, I don't have a date for when that work is going to start, but I can post an update when I know more. What do you think? -- DannyH (WMF) (talk) 22:04, 27 October 2017 (UTC)[reply]

I suggest that the appropriate response to the continued refusal of WMF to stop injecting Wikidata text into Wikipedia by default (evident above) is to write a bot that makes sure every Wikipedia article has a non-blank magic word. It could be generated automatically from the first sentence as suggested in earlier discussions; getting accurate text for the bot-generated description is less important than getting something there, to override the use of Wikidata everywhere and bring this to the attention of human editors who can fix bad descriptions. —David Eppstein (talk) 22:47, 27 October 2017 (UTC)[reply]

I think this is a reversal from what you said last time.

I think the WMF is once again asserting Wikidata as the only option.

I think we have allowed this stonewalling to drag on far too long.

I think I'd like to modify David Eppstein's proposal above. If the WMF refuses to even discuss any option other than Wikidata, if the WMF escalates saying the magicword-override will even disregard blank descriptions, then we shouldn't have a bot apply non-blank descriptions. If the WMF wants us to edit our pages via Wikidata, fine, let's do it properly. We should apply our blank descriptions at Wikidata itself. If the Wikidata community gets pissed off and blocks us, I'm OK with that. All the better to help get Wikidata banned entirely from Wikipedia. Alsee (talk) 10:57, 28 October 2017 (UTC)[reply]

... or you could be constructive by adding useful descriptions to Wikidata? Thanks. Mike Peel (talk) 11:26, 28 October 2017 (UTC)[reply]

As I'm sure you're aware, consensus was against Wikidata descriptions. Perhaps to should ask the WMF to be more constructive.

P.S. to my original comment, I withdraw my tepid statement of support for "Wikidata-default with an if-present magicword override". I was trying to reach for a compromise the WMF would accept, but it's a dumb design. We shouldn't have to run a bot to enforce a magicword on every page in order to suppress a problem-default pushed out by the WMF. ^{Nevermind that part. I think I was mis-remembering. I considered the option, but I don't think I expressed tepid support for it.)} Alsee (talk) 11:38, 28 October 2017 (UTC)[reply]

DannyH, I don't know whether you are just the messenger or the actual force behind this proposal, and I really don't care any longer. This is just the same as what happend with Flow, Gather, ... and your role seems to be the same as with the Flow debacle. I don't know whether you don't care about what enwiki thinks and still believe that the WMF can get away with anything and everything, or that you simply are incompetent, but please let this be the last time that you work "with" us, as you are simply a total waste of our time and patience. Why do you and the WMF continue to insist that Wikidata is the default? NO! This is language-based content which should be curated here and only here. Using Wikidata may have seem like a good idea from the WMF ivory tower and from the idea that Wikidata is the future for everything, but in reality it isn't, and this long discussion should have made that abundantly clear by now. Every time you came with this same idea of using Wikidata as the efault, it has been shot down, so your final solution is to use Wikidata as the default? Get lost. All you are creating is more aversion against both the WMF and Wikidata, as if either of those was needed.

I'll address just one blatant fallacy in your supposed justification: "When a new item is created on Wikidata, there's a strong call to action in the UI to write a description". Tell that to the bots who create most of the new items on Wikidata. Apart from the "scientific article" ones which will for 99.99% never have an enwiki article anyway, we get all kinds of persons created (without a description) by "QuickStatementsBot"[4][5][6], "Emijrpbot"[7][8][9], and so on. Note that the persons created by Emijrpbot already have an enwiki article, so here the inaccuracy of your claims is blatantly obvious. "Real" editors sometimes do use the description, but when some of the most fanatic pro-Wikidata editors don't bother adding a description[10] then even this segment is not really corresponding with your hopes. Oh, and perhaps someone should block this bot which has created the same Wikidata item five times in a row (and a sixth time a while ago, and something like this which is a duplicate of this. Basically, Wikidata has highly insufficient editorial oversight, vandal fighting, consistency, policies, ... which is obvious any time you look at what happens there.

In general, enwiki shouldn't outsource English language content maintenance or production to other sites anyway; in particular, enwiki shouldn't outsource it to Wikidata. So why, after all this, is this still what you propose? Fram (talk) 09:53, 30 October 2017 (UTC)[reply]

I got involved in the last stages of the deprecation of persondata, that is the stage where we were hammering out a system to actually completely remove all persondata from mainspace. I rather tried to slow it down, at least impede an uncontrolled deletion, thinking, maybe we might need this still some day. But the reasons for removal were overwhelming, and thus we eradicated uncountable hours of work by thousands and thousands of Wikipedians. Sure, the system that is proposed now will be "different" in several respects, but reasons for not having it will be, in whatever form we have it, still overlap to a great extent with the reasons for eradicating the persondata system (and these reasons were sound, nobody doubted that).

Long story short: I don't think we need a system that knowingly or unknowingly mimics the old persondata (again, even when several people will contend it is "different" in many respects). There are too many reasons for not having it, without even speaking about the insult to the Wikipedia editing community: first they were asked to provide and maintain data for a system that on most interfaces didn't show up; then the content in that system was completely and utterly eradicated, flushing countless man-hours down the drain; and now there is talk about embarking on a similar adventure – to which my answer would be: no, no and again no, whatever we do, not something that has too many similarities to that debacle. --Francis Schonken (talk) 08:38, 1 October 2017 (UTC)[reply]

It was extremely annoying, given the amount of work I had put into systematising alternative names, in particular. I'm not sure if the data is stored anywhere retrievable. All the best: Rich Farmbrough, 19:29, 18 October 2017 (UTC).[reply]

@Rich Farmbrough: There is a wikidata game that uses the info from persondata, which implies that the data is still available. Thanks. Mike Peel (talk) 19:31, 19 October 2017 (UTC)[reply]

A discussion here put me to thinking: what is the difference between the descriptor and a dictionary definition? Should the Wikidata descriptor be identical to the Wiktionary definition for a term that appears in that dictionary? Does the policy Wikipedia:Wikipedia is not a dictionary come in play when what we're obviously trying to do here is "enrich" Wikipedia (an encyclopedia) with short dictionary-like definitions? --Francis Schonken (talk) 08:23, 21 October 2017 (UTC)[reply]

A dictionary type definition as part of an article is not the same as a dictionary type definition as the article, but there are only some articles for which a dictionary type definition is appropriate, in some cases even possible, However, I think that is what we should aim for as the descriptor.· · · Peter (Southwood) ^(talk): 20:44, 27 October 2017 (UTC)[reply]

the override feature is worthless / Related Pages / WMF putting images on articles

Above Fram asked we decide which image, if any to use, no images are sent by default ever (I hope, or has the WMF done this as well somewhere?)

File:Andrex_puppy_(1994_advert).jpg: The Andrex Puppy, seen here in a British advertisement from 1994.

Unfortunately the answer is YES. In fact it took me two minutes to find that the WMF literally slapped an advertizement for a specific brand of facial tissues on our Facial tissue article. You can (maybe) see the problem at the bottom of the mobile view of Facial_tissue. However the Related Pages feature selects "see also" articles (and images) dynamically, so you may see different article links with different images. But if you randomly go to articles on "product" type items, it shouldn't take long to find one with a specific brand being promoted on the page. The Related Page feature links to articles containing many of the same words, so companies that sell that product are likely to be selected. You can also get some seriously bad BLP violations.

The community previously complained about this issue, but the WMF apparently didn't consider the concerns important. Not even when people were finding serious BLP violations on biographies. (I think one politician was linked to fascism, and if I recall correctly, Pig-faced women appeared somewhere bad.) They decided the "solution" was to hide the feature from editors hide the feature from desktop view, and to offer the same worthless "override" solution being proposed here. The Related Pages that are given change pseudorandomly, so only way the override feature would actually fix the problem is if we run a bot to set three (blank) overrides on EVERY every article.

Returning to the original topic: If the WMF insists on the "override keyword" for wikidata descriptions, then I now propose running a bot to apply the override on every article. The bot can either copy the lead sentence, or leave it blank with a hidden comment saying to fill it it. The same bot may as well apply three Related Pages overrides at the same time. Alsee (talk) 20:09, 29 September 2017 (UTC)[reply]

Hang on, didn't this note that non-free images would be excluded from that feature? That Andrex image is tagged as non-free - quite apart from advert concerns, it shouldn't be showing up there. Nikkimaria (talk) 20:27, 29 September 2017 (UTC) @Deskana (WMF): seems to have been active on the phab task linked. Nikkimaria (talk) 20:33, 29 September 2017 (UTC)[reply]

Nikkimaria, I'm pretty sure that appending a ping to an existing comment like that doesn't work. If you look at the diff for your edit you'll see that the blue diff-text does NOT contain a proper timestamp. Pings are only sent on a signed edit, and the software doesn't see a proper signature in that diff. I'll add your intended ping to @Deskana (WMF):. Alsee (talk) 21:09, 29 September 2017 (UTC)[reply]

@Nikkimaria and Alsee: I don't work in this area any more, but I took a look and managed to fix it. See my comment below. --Dan Garry, Wikimedia Foundation (talk) 10:09, 30 September 2017 (UTC)[reply]

Yes, quite aside from the advertising issues, this is a clear violation of our image use policy, as we have no fair use rationale for this use of the image. —David Eppstein (talk) 20:41, 29 September 2017 (UTC)[reply]

Holy crap, it didn't occur to me that this was a non-free image. WTF?! I saw the Phab task that supposedly prevented that.

Also, I had added the image to this section without noticing the non-free issue. I tried to remove it, but edit conflicted with Ymblanter fixing it. (Thanks Ymblanter.) Replaced with link to file page. Alsee (talk) 21:01, 29 September 2017 (UTC)[reply]

Apologies for reverting the second time, I misread the diff. Now I restored it.--Ymblanter (talk) 21:07, 29 September 2017 (UTC)[reply]

(ec)I thought phab:T124225 was resolved with the result that our non-free content policy won against the wishes of some of WMF's various coding teams. Has anything changed there? —Kusma (t·c) 21:09, 29 September 2017 (UTC)[reply]

The clear attitude on display here by WMF is that whenever we come to a decision or consensus that goes against the wishes of the coding teams, they will go ahead and follow their wishes anyway. I can predict that their response will be that the phab thread only talked about "mobile apps, RelatedArticles, Gather, and mobile web search" and that the related article links on the mobile view are something other than these things and therefore not covered by this decision. —David Eppstein (talk) 21:21, 29 September 2017 (UTC)[reply]

I am disappointed but not surprised by the WMF's response on the Wikidata descriptions... however I bet this non-free image issue is either a bug or (at worst) someone unaware who made an undiscussed change. I get the impression that WMF staff were OK with the non-free image issue, they explicitly built code to restrict when non-free images would be returned. (There is explicit concensus that non-free images can appear in the Popup-article-preview feature.) Alsee (talk) 22:34, 29 September 2017 (UTC)[reply]

@David Eppstein: It's a bug caused by a bad template. Please don't jump to conclusions. --Dan Garry, Wikimedia Foundation (talk) 10:09, 30 September 2017 (UTC)[reply]

I just created a Phab listing for the non-free image appearing in RelatedPages. Alsee (talk) 22:20, 29 September 2017 (UTC)[reply]

I don't work in this area anymore, but I took a quick look. The non-free page image associated with the Andrex article was due to an issue with Template:Non-free television screenshot which was not tagging the images associated images as non-free. I fixed the issue with that specific template. Maintenance of content-related templates is normally outside Foundation jurisdiction, so I encourage other people to take a look and see if the problem exists elsewhere. This is why we need structured data on Commons, by the way; this problem never would've happened if the data were properly structured. --Dan Garry, Wikimedia Foundation (talk) 10:09, 30 September 2017 (UTC)[reply]

@Deskana (WMF): (a) we do have such a data structure, the coders just chose to use something else instead of our Category:All non-free media. (b) Wouldn't it make more sense to fix the issue by adding the invisible code you're looking for to Template:Non-free media instead? Adding specialised code to lots of individual non-free licensing templates seems not the best way to do this. —Kusma (t·c) 16:19, 30 September 2017 (UTC)[reply]

The Non-free media template is doing that already. This is just a conflict between two templates. Dan made an intermediate fix, but that fix should not be necessary long term. I suggest people stop fingerpointing and stop blowing everything up. No one is able to keep up with what is going on right here, it fires of into every direction, this whole discussion is becoming completely useless this way. —TheDJ (talk • contribs) 17:26, 30 September 2017 (UTC)[reply]

Deskana (WMF), thanks. I suspected this was some unintentional flaw. However I'm extremely surprised and confused at the method the WMF is using: an invisible, redundant, and apparently-undocumented-on-EnWiki "span" class. Is there any chance that the WMF could use the (obvious) Category:All non-free media? That's how we track non-free files. I expect more files will continue to slipping through the cracks based on the span method. Alsee (talk) 02:05, 1 October 2017 (UTC)[reply]

Hi User:DannyH (WMF):

You wrote here about we want to give English Wikipedia control of the descriptions and here about the people at the WMF getting more nervous about community control.

Control over en-WP content is not WMF's to "give". If WMF staff are "nervous" about the fundamental deal in the movement, then you should train them better. Control over en-WP content published by WMF as "Wikipedia" resides solely with the en-WP editing community. That is the fundamental deal in the movement.

The sprawling conversation above is exactly the kind of thing we get at AN/ANI when someone is being seriously out of step with community norms and we end up with drama exploding in all kinds of directions - when people are persistently doing what they can do, and not what they should do, in big ways. In this case it is more explosive because it is not a person, but an organization. In this case is is people using Wikidata in ways that it can be used, but not using it as it should be used in en-WP, in ways that respect the policies and guidelines here.

This conversation would be totally different, if it were being conducted on appropriate foundations - if it were 2015 and the WMF had come to en-WP and said - "Hey short descriptions would be really useful in a bunch of ways. We have a field in Wikidata we could use, but it would be best if en-WP content came from en-WP. How can the en-WP community help out here?" That is an entirely different conversation. That is an appropriate conversation.

We can still have it. There are people here willing to try to work within the en-WP community and in conversation with the WMF to generate en-WP native short descriptions that will be useful to WMF, but the conversation must happen on appropriate foundations that respect the respective roles of WMF and the en-WP community within the movement.

Please:

1. Acknowledge that WMF overstepped in unilaterally and systematically changing en-WP content.
2. Express an understanding that if the WMF wants to do this kind of thing in the future, it will get consensus beforehand
3. Remove the Wikidata descriptions from the app content.

The WMF is a creating a sort of constitutional crisis that is entirely unnecessary. Rushing ahead to solve the problem, without dealing with what has caused the problem, is foolish. We should never be in this situation again.

Please reply and say yes to the three things above. Please. Jytdog (talk) 19:38, 2 October 2017 (UTC)[reply]

Hi, Jytdog. I agree with you in a lot of ways. I think there should be way more communication and sincere partnership between the WMF product teams and the communities. That's something that I've been working on for a while, as the PM for Community Tech. I agree with you about what the 2015 conversation should have been, and I'm currently talking with people on the WMF product team about how we get closer to that, as an organization. That's why they asked me to be the lead on communication here, so that I can help us move in that direction. It's a process. So I wouldn't use the same words that you do, but yeah, this should have been done better in 2015, and I'm trying to help it be done better now.

For #2, I have a similar answer. We're working on it. I know it doesn't mean very much when I say that; the thing that matters is what we actually do, from now on.

For #3, I've answered that before. Short descriptions are very useful for the app. We're currently talking with folks on this page about how to help transition from a Wikidata-controlled system to an English Wikipedia-controlled system. During these conversations and while we're implementing the solution that we agree on, we're not going to take the existing descriptions down, because that would damage the users' learning experience, for the sake of making a point. I don't think that's necessary, or a good thing to do.

I know, my answers are more complicated than "yes", and that's not what you asked for. But these are my honest answers, if that helps. -- DannyH (WMF) (talk) 02:42, 3 October 2017 (UTC)[reply]

How much can you be "working on" item 2? We want you to agree to follow the rules and you are "working on it"? That does not sound good faith, it sounds like you are weaseling for loopholes or stonewalling. · · · Peter (Southwood) ^(talk): 06:10, 4 October 2017 (UTC)[reply]

@DannyH (WMF):. That's not acceptable. If the subject of a biography (BLP) objects to being labelled as "Jewish", for example, or any other contentious ethnic/religious/racial categorisation, and there is no strong sourcing to support that label, then our policy is that it "should be removed immediately and without waiting for discussion" – WP:BLP, quoting "WikiEN-l Zero information is preferred to misleading or false information" Jimbo (@Jimbo Wales: FYI). Have you forgotten the Wikipedia Seigenthaler biography incident already? When the information is curated on Wikipedia, we see changes on our watchlists, and can insist on sourcing. There is no sourcing for Wikidata labels. It is a fundamental mistake to draw possibly contentious data from a source that is incapable of showing a reference, and it's stupid mistakes like this that makes it difficult for any of us who are working on legitimate ways of using Wikidata in the English Wikipedia. You will take the existing descriptions down – despite what you think about "users' learning experience" – because sooner or later, you'll put the WMF in the position of being sued by an aggrieved BLP subject and I'll make sure that they know you were warned about the potential danger and chose to do nothing. --RexxS (talk) 14:43, 6 October 2017 (UTC)[reply]

@RexxS: Hold on. I thought there was consensus that sources were not required for infoboxes or the above-the-content lead section if the material was supported by sources in the main body of the article. It is straightforward for editors to check if there is sourcing for the claim in question in the rest of the article. (Or, for that matter, in an appropriate statement on Wikidata). The Wikidata description can be changed right now if we got a complaint, and I am sure WP:OFFICE would not hesitate in doing so, if a complaint came in. So I can't see any danger of any problem that would be outwith Section 230. The proposed opt-out mechanism would allow an even more direct approach -- it would make it very straightforward to put in an override description here, and lock it. Jheald (talk) 20:35, 6 October 2017 (UTC)[reply]

@Jheald: With all due respect, I don't think I'll be holding on. Please look at the description field of any article on Wikidata. There is no guarantee that a source for that is present on English Wikipedia. Nor is there any way for Wikidata to store a source for any description. I'm not talking about editors; I'm talking about readers. How would the average person who reads a BLP about themselves on the Wikidata app and see themselves labelled as something they strongly disagree with go about editing that? Have we forgotten already about the alt-right nutjobs who go around adding Category:People of Jewish descent to everybody they didn't like? What if they find Wikidata's description field? How long before we see Bernie Sanders described as "Jewish-American Politician"? It's anything but straightforward for 90+ percent of Wikipedia editors to find where the description on the app comes from, let alone someone who doesn't even edit. How would they even know where to complain? I'm sorry, but that's just not good enough. We must use the precautionary principle in our descriptions of living people, and text which has no chance of being verified doesn't belong in the second line of every BLP displayed on the Wikipedia app. Nor, for that matter, does the decision on whether text is displayed in any version of English Wikipedia belong with a staffer, rather than the community. Especially when it's directly in violation of our BLP policy. --RexxS (talk) 21:10, 6 October 2017 (UTC)[reply]

With luck, we have pointers to direct concerned individuals to WP:BLP/H, or WP:BLP/N, or the article talk page. If you're worried that it's too hard for them to find WP:BLP/H, then that's a different question: what we can do to make WP:BLP/H better signposted or easier to find.

On the "difficulty to edit" point, that is precisely what the latest updates to the system were designed to make easier -- direct editability of the description from the app, or (in future) from desktop Wikipedia. (CSS patches may even already be available for this, for wikis currently supporting the broad use of these descriptions)

As to your point about unverified text in the second line -- well, at the moment there is no requirement for inline sourcing for text in the second line of desktop Wikipedia, as I said above, so long as it is supported by the rest of the article.

It's also quite a slide to go from "There is no guarantee that a source is present" to "no chance of being verified".

Besides, if one did want to create a requirement for sourced support before particular keywords were allowed in the description, it would actually technically be much easier to implement on Wikidata, where an automated script could look for an appropriately sourced supporting statement much more easily than having to parse the natural language of a complete page of wikitext. Jheald (talk) 00:06, 7 October 2017 (UTC)[reply]

@Jheald: Why do we have to trust to luck to avoid another Wikipedia Seigenthaler biography incident? I'm now looking at a BLP on the Wikipedia app. Please tell me how I find these pointers on the article talk page. Or even find the article talk page. You can't.

Where is the direct editability of the description from the app? Looking at a couple of BLPs, no matter what I try, I just get "Sorry, this page cannot be edited anonymously at this time". That's a big help to anybody looking at their own BLP. Ok I found one that an unregistered editor could edit: Billie Jean King American tennis player. How would I edit "American tennis player" if it changed to something I found contentious? You can't.

The desktop version of Maria Callas doesn't have "American-born Greek operatic soprano" as its second line, or anywhere else (the app does!). Are you sure you understand where this problem is occurring? Callas held both Greek and American citizenship for most of her life, so the description "American-born Greek" is contentious, by any standards. Not to mention wholy unsourced.

It wouldn't be any easier to implement on Wikidata than on Wikipedia, because neither you nor I (nor the devs) are capable of writing an algorithm that detects contentious statements where there is no sourcing available to test them against. Even if you could, how would the fact that this magical script has found appropriate sourcing be recorded? There's nowhere in the Wikidata entry to store a reference for the description, much less the fact that a script had found one. --RexxS (talk) 00:41, 7 October 2017 (UTC)[reply]

@RexxS: To flesh out what I was suggesting, if there was a list of words one was sensitive about in the description -- eg "Jewish" -- one could test for whether there was a supporting statement on the item with an appropriate value, eg religion or worldview (P140) = Judaism (Q9268) (or a subclass of it), and what kind of source (if any) that statement had on it. One could then use that to auto-classify whether the word "Jewish" in the description on the face of it appeared potentially acceptably supported, not well enough supported or unsupported.

But that would be for the future. My more fundamental point, is that if we look at eg Billie Jean King, there is no sourcing in the first line for "is an American former World No. 1 professional tennis player". But it is acceptable, without specific inline cites, because it summarises facts in the rest of the article, that are required to support it. Similarly Maria Callas begins "was a Greek-American soprano", with no inline cite to back up that. Yes, the article doesn't begin "American-born Greek operatic soprano", but somebody could have edited that in, and they wouldn't have been required to provide an inline cite to back it up. So why make such a fuss that a definition pulled from Wikidata might say that and not have the equivalent of an inline cite? The two cases seem directly parallel to me.

As to the direct editability, my understanding was that it was the addition of this feature on the Android app that sparked this current discussion. Jheald (talk) 01:07, 7 October 2017 (UTC)[reply]

All words are potentially problematic, machine intelligence is not up to the task of evaluating whether the sourcing is adequate to justify specific wording, and WP:BLP is not optional and something we can put off "for the future". —David Eppstein (talk) 01:59, 7 October 2017 (UTC)[reply]

No, but we be able to identify that a source is one the potential to be adequate, even if we can't guarantee that the actual text is supported.

To get the descriptions we're going to have to start from somewhere (and we need them right now in searches). We realistically aren't going to build 5 million, or even x hundred thousand BLPs, starting over for all from scratch. It's not helpful to say you wouldn't start from here, when here is where we are. What is more helpful is to think how to move forward from here, in a prioritised achievable way. Jheald (talk) 02:15, 7 October 2017 (UTC)[reply]

• @DannyH (WMF): you may recall the unease that followed President Trump failing to mention the Jews on International Holocaust Remembrance Day because, the White House said, other groups had suffered too. [11] The WMF description follows suit, with "programme of systematic state-sponsored murder by Nazi Germany". It isn't wrong, but it's the product of one person on Wikidata deciding in 2013 to override the academic debate about the definition of the Holocaust, without anyone on the English Wikipedia realizing they had done it. (Please don't reply that therefore we must watchlist Wikidata; no one wants to be forced to take part in another project.) Before the change, the Wikidata description was: "Nazi German genocide of approximately six million Jews during World War II." [12] (It could be worse; at one point it said: "The 6 million Lie".) [13] SarahSV ^(talk) 22:00, 6 October 2017 (UTC)[reply]

Worth noting that "6 million lie" was reverted within 19 minutes, back in March. Since then anti-vandalism has continued to improve. Increased visibility, less flooded watchlists, watchlist-based reversion, and better integration with en-wiki tools should all help further, for those that do want to do anti-vandalism.

Also worth noting that the description currently reads "state-sponsored genocide of Jews by Nazi Germany", even if only since last night, and even if as you note some might see this as an overly narrow definition. Jheald (talk) 23:30, 6 October 2017 (UTC)[reply]

Besides, the proposed override mechanism addresses exactly this, allowing en-wiki to supply hand-crafted descriptions for the highest value and most sensitive articles, without requiring such measures for the multitude of run-of-the-mill stubs. Jheald (talk) 23:44, 6 October 2017 (UTC)[reply]

Ymblanter changed it 13 minutes after I posted here. [14] (Jheald, I didn't say anything about an overly narrow definition.) The point is not what it says, but that one person on another website can override the debate. SarahSV ^(talk) 23:54, 6 October 2017 (UTC)[reply]

(ec) You noted that there was "academic debate" as to the most appropriate definition. At The_Holocaust#Terminology, where we review that debate, we review three potential definitions presented by Gray; and that Niewyk and Nicosia preferred a broader definition. You noted there is a debate; the position of some in that discussion is to prefer a broader definition.

As to the proposition that "one person on another website can override the debate", this is precisely what the suggested "local override" option addresses, putting ultimate control back in the hands of en-wiki. Jheald (talk) 00:21, 7 October 2017 (UTC)[reply]

More importantly, how is an patroller to know that a particular description may or may not be contentious? The announcement of this idea used two screenshots of Maria Callas at mw:Reading/web/Projects/Wikidata Descriptions. The potential problem was explained (and ignored) on 26 August 2016 on the talk page mw:Talk:Reading/web/Projects/Wikidata Descriptions:

[Melamrawy (WMF):] "In that specific example, why would the "American born Greek" description be problematic if Maria Callas was alive?"

[RexxS:] "Because completely unsourced descriptions of a living person's ethnicity are potentially offensive or libellous, or both. Callas was an American of Greek descent and may have rightly objected to being identified otherwise without any supporting evidence. The idea that we might be using somebody's unverifiable judgement of a person's identity is so far from WMF's policy on living people that you really ought not to have to ask why it's a bad idea."

Nothing I've seen since has led me to believe that WMF staff have any better understanding of "Contentious material about living persons ... that is unsourced or poorly sourced—whether the material is negative, positive, neutral, or just questionable—should be removed immediately and without waiting for discussion." Nor have I revised my opinion that somebody's unsourced opinion (the Wikidata description) is acceptable in any way, shape or form as a description of a living person. No source = no inclusion on enwp. That's not negotiable. --RexxS (talk) 00:12, 7 October 2017 (UTC)[reply]

And yet, we allow free summarisation in the lead paragraphs of an article of the rest of that article. Don't we? Jheald (talk) 00:25, 7 October 2017 (UTC)[reply]

Jheald, I believe you said something above about leads not requiring sources, but they do for anything contentious, including BLP issues. WP:LEADCITE: :"Any statements about living persons that are challenged or likely to be challenged must have an inline citation every time they are mentioned, including within the lead." And "Complex, current, or controversial subjects may require many citations; others, few or none." SarahSV ^(talk) 01:29, 7 October 2017 (UTC)[reply]

Concerning vandalism on wikidata, notably on WP:BLP, Suga entry on Wikidata is being the subject of unabashed IP vandalism for many days already, without nobody apparently noticing it, with the vandal editions alternating with regulat botlike editions, messing up the whole thing. I was testing an Infobox using Wikidata information on that article at the Portuguese Wikipedia, but had to remove it from there. Not only the Wikidata entries do not seem to be properly monitored at all, they stay subject to vandalism/edit wars after the wikipedia article is protected (as was the case with Suga). The presence of the wikidata infobox there, with direct links to easily edit the Wikidata entry, was also apparently serving as a magnet to attract vandalism from our project straight into the wikidata entries, which are off-limits of the wikipedia watchlist (and which I'm not interested watching, anyway, I already have plenty of things to follow in my regular projects). As it is now, I do not consider Wikidata capable of being used on WP:BLP Infoboxes, at all.-- Darwin ^Ahoy! 03:57, 7 October 2017 (UTC)[reply]

Wikidata values appearing in Wikipedia are effectively bot edits

Various discussion above addresses whether or not Wikidata values should only be included when sourced, or subject to other restrictions or limitations.

Most editors on Wikipedia are familiar with things like BLP issues, what sort of information should be included where, and what does or doesn't need to be sourced. For example information in an infobox doesn't need a source if the information is sourced in the body of the article. We allow new users to edit without learning all of those rules first. We allow that as an expected and accepted part of teaching new contributors how to edit here productively.

Bot-edits are subject to far stricter scrutiny than human edits. These kinds of automated mass-changes are expected to affirmatively demonstrate (1) the clear positive value of the class of edits, (2) very high scrutiny on any error rate that might downgrading existing content or require human cleanup, and (3) strong observance of any relevant policies such as sourcing requirements.

Wikidata values appearing on Wikipedia are effectively bot-edits build directly into the Wikimedia software. A staggering proportion of edits at Wikidata are bot edits, then software is preforming a bot-like transfer of that bot-edit edit to Wikipedia.

Where Wikidata edits are preformed by humans, the software still preforms a bot-like transfer of that edit to Wikipedia. Neither the wikidata edit nor the transfer to Wikipedia are effectively subject to Wikipedia policies. The Wikidata edit is often done by people who have no knowledge of Wikipedia policies, and we are unable to educate/integrate that person to make more appropriate edits in the future. If a new user makes an edit here and we have to clean it up, we're investing in teaching that user how to edit here. If we have to clean up an edit that was bot-transferred to Wikipedia, we have to keep cleaning up future edits. Even if we go to wikidata and clean it up there, even if we contact that user on wikidata trying educate them what kind of edits we want, wikidata isn't subject to our policies. (Not to mention the fact that there's no reason to expect that user to speak English.)

If we look at wikidata-on-wikipedia as bot-edits, it becomes clear that the concerns raised here aren't new. Wikidata isn't facing random attacks, it isn't being baselessly held to random higher standards by haters. If the software for displaying wikidata-on-wikipedia were separated out, if it were a bot making edits to copying these values onto wikipedia, if that bot were submitted as a routine bot-approval request, I think it that bot-approval request would go down in flames. Alsee (talk) 14:56, 8 October 2017 (UTC)[reply]

I do not quite understand your allegations. There is a process on Wikidata on bot task approval, quite similar to the English Wikipedia process, subject to the community scrutiny, and as a Wikidata crat I personally approved most of the requests. Currently, the vast majority of bot edits are actually transfer of data from reliable databases, which provide sources and would even satisfy the BLP policies of any Wikipedia.--Ymblanter (talk) 15:08, 8 October 2017 (UTC)[reply]

Ymblanter, I'll try to explain it more simply and explicitly for you.

1. Someone goes to the wikidata item for city, and adds an unsourced statement that it is: capital_of country.
2. The Wikidata community then approves a bot to create a massive number of reciprocal statements.
3. That bot sees city says it is capital_of county. It copies that info into the wikidata item for county, saying that country has capital city.
4. The bot helpfully adds a reference for that edit: Stated_in Q(city). Note that this is a circular reference, saying this information was sourced from the wikidata item for city.
5. The wikimedia software then copies that information from the wikidata to the EnWiki infobox for county stating the capitol is city.

I am saying that step 4 is functionally a bot edit, copying the information from wikidata to wikipedia. And as an added bonus, independent of the "bot" issue, this bypasses the filter against "unsourced or wikipedia_sourced" information. It was unsourced information when it was added to Q(city), and it did not magically become sourced when it was copied to Q(county). And it was effectively a bot that copied the unsourced information into wikipedia. There are a massive number of unsourced/wikipedia_sourced statements with WP:circular-references bypassing the "sourced only filter". I identified over a million wikipedia-sourced statements bypassing the filter[15], and I can't even begin to determine how many circular "Stated in: Q(other wikidata item)" are bypassing the filter. All I know is that humans and bot-runs have been creating them in large numbers. I ran into a whole pile of them in just a few minutes of browsing an arbitrary list of wikidata items. Alsee (talk) 10:59, 20 October 2017 (UTC)[reply]

No, I do not think we approve such bot tasks, at least definitely not now.--Ymblanter (talk) 11:07, 20 October 2017 (UTC)[reply]

I don't know what bots are being approved at this very moment, however the proposal to create "inferred from"[16] explicitly states such bots were "currently" running as of a year ago: Currently we have a bots that automatically add inverse statements. They sometimes use "stated in". I think it would be worthwhile to have a more specific property for the use case. ChristianKl (talk) 10:07, 27 September 2016 (UTC). And as I said, I came across many of these WP:circular references in just a few minutes when I was randomly skipping though Wikidata items matching EnWiki articles bearing a specific template. So these are far from rare. Alsee (talk) 13:35, 20 October 2017 (UTC)[reply]

Wikidata values appearing in Wikipedia are not effectively bot edits

Most of the 47,518,445 registered users are not familiar with things like BLP issues, what sort of information should be included where, and what does or doesn't need to be sourced. Judging by the number of times we have to clean up mistakes, most of the 118,638 active users aren't particularly familiar with those issues either. Here's an example: with very few exceptions, all information displayed in an infobox must also be included in the body of the article, which makes the information subject to the normal sourcing rules, with the caveat that we often don't normally repeat citations for summaries such as the lead or the infobox (although it must be sourced if challenged even in the lead or infobox). We have editors who have been editing since 19 September 2006 who don't know that.

Wikidata values displayed on Wikipedia are passed through a filter which rejects unsourced data. In that way, they are better sourced than statements made directly on Wikipedia, where the local editor may choose, deliberately or through ignorance, not to include a source. Of course a cite on Wikidata may be bogus, but that applies equally to a cite made directly on Wikipedia. At least when the data comes from Wikidata, we are able to guarantee it's sourced. That means that we insulate Wikipedia from whatever is used to provide the data to Wikidata: if a bot or human adds a source, we get sourced data; if the bot or human doesn't add a source, we don't let it into the infobox. There is nothing bot-like about the process of fetching Wikidata into Wikipedia. The code is written by a dinosaur, not a bot. Whatever policies need to be met, we can write the software to meet them, or make it easy for editors to amend any inaccuracies that they spot in the value returned from Wikidata. When we fix mistakes on Wikidata, we don't just improve things for us, but for all the other 288 active Wikipedias.

We have the ability to make any infobox Wikidata-aware without disturbing a single article already using that infobox ("opt-in"). We can give the editors who curate a particular article the ability to switch on or off the fetching of data from Wikidata, in the infobox as a whole or field-by-field. We can provide links from the infobox directly to the statement on Wikidata where the information is fetched from. With that degree of control available in a suitably designed infobox, there's no doubt that if the Lua modules that actually do the fetching were subject to the same sort of scrutiny as bot applications, they would be passed with flying colours. --RexxS (talk) 17:36, 8 October 2017 (UTC)[reply]

I don't agree with you here RexxS and I ask you not to obscure the underlying issue, that enabling Wikidata in an infobox and then deploying that infobox widely is way more "bot"-like than any kind of normal editing -- a) the coding to set up these infoboxes is advanced, like bot programming and not something in the realm of non-coders (i haven't invested the time to learn, and am not interested at this time in doing so); and b) it does exactly allow changing one thing (one data field in Wikidata, or one bot run in Wikidata) to change many articles in Wikipedia, which is way more bot-like than normal editing. Enabling something like should be thought through and approved before release infobox-by-infobox before they are deployed.

And please don't use this old saw about "go fix it on Wikidata"; since you have, the standard responses to that remain the same as always -- a) WP editors are not necessarily volunteering to edit Wikidata (some may; I for one don't); b) this sets up a situation where WP editors need to go to Wikidata to fix policy violations or mistakes, which is essentially blackmail and hijacking of WP volunteer time; and c) driving WP editors to go edit WD just sets up intra-project edit wars between projects that have different policies and guidelines. Jytdog (talk) 11:52, 10 October 2017 (UTC)[reply]

Advanced coding is not the same as bot work. I agree that there are potential issues around differing policies, but I think that particular point is the one obscuring the underlying concerns. Nikkimaria (talk) 14:24, 10 October 2017 (UTC)[reply]

RexxS wtf? If a human adds unsourced information to an infobox, that's not a bot edit. That is a newbie, and we are willing to invest cleanup work in the hope of them making better edits tomorrow. If a piece of software copies unsourced information (or copies sourced information) from wikidata into wikipedia, that is a bot edit. The fact that someone at WMF shoves that bot into the wikimedia software itself doesn't change anything.

Regarding your comment Wikidata values displayed on Wikipedia are passed through a filter which rejects unsourced data, that filter is doesn't work. I just found two vast classes of wikidata statements bypassing that filter. There are over a million wikipedia-sourced statements were bypassing the filter via a tools.wmflabs ref. Those could plausibly be added to the filter. I found an unknown but vast numbers of unsourced statements bypassing the filter via circular refs "Stated in: Q(other wikidata item)". Not only have humans been creating those circular refs, there were bot runs creating them en-mass. I invite you to add that to the filter - trying to filter them would pretty much nuke the import of most other wikidata statements. But more importantly, all of this highlights the utter worthlessness of the "sourced-only" option in general. Those are obviously not the only problems. The wikidata-community has jack-squat expectations for sourcing or anything else. Alsee (talk) 11:22, 20 October 2017 (UTC)[reply]

Wikidata has been removed from recent changes in Commons and ruwiki, will be removed from a dozen other wikis (including Italian, French, and Swedish), but remains enabled on enwiki for the time being.

Fine, but it is utterly useless here. At the moment, I have to restrict recent chanegs to mainspace only, and manually override the limits to show 5000 edits instead of the defualt maximum of 500, to get some Wikidata changes. Basically, when I ran "recent changes" at 08.01, the most recent Wikidata change displayed is from 07.32, or half an hour old. For recent changes, this is utterly pointless (the standard "max" of 500 changes only goes back 10 minutes now), and makes recent changes on enwiki useless to catch Wikidata vandalism.

This isn't the first time this has happened, in my experience the delay fluctuates between 2 minutes at the very best to 24 hours at the worst.

As long as this problem occurs, every claim of "but you can check Wikidata changes from your watchlist and recent changes" becomes basically void, as it means that most changes won't be seen, or at best will be seen much too late. Fram (talk) 08:15, 17 October 2017 (UTC)[reply]

Delay now nearly an hour (09.54, most recent Wikidata change 08.58). The same applies to the Watchlist, but is of course less obvious there because it depends on having a recent Wikidata change to an item on your watchlist. Fram (talk) 09:58, 17 October 2017 (UTC)[reply]

That is fairly terrible. If this isn't fixed soon (I hope it is), we will have to stop accepting data from Wikidata. Or at least apply the same time delay to updates from Wikidata that our monitoring feeds have. —Kusma (t·c) 11:31, 17 October 2017 (UTC)[reply]

"will be removed from a dozen other wikis" - citation needed. If you want to catch Wikidata vandalism immediately, then you can always use the recent changes page on Wikidata. For most cases of watching changes to the content of infoboxes etc., then a delay of this kind of length isn't vital (particularly as pages are cached, so any changes from Wikidata might not show up immediately anyway unless the page is purged). You can see the dispatch stats on Wikidata at d:Special:DispatchStats. Mike Peel (talk) 12:10, 17 October 2017 (UTC)[reply]

You don't understand it, do you? First of, for the dozen further wikis where this will happen, you could look at the relevant phabricator ticket, where dba jcrespo has today announced:

"Given the above data, I would like to disable recentchanges from wikidata (temporarily), purge and defragment on the following wikis:

bgwiki itwiki svwiki zhwiki bewiki cewiki dawiki hywiki ttwiki frwiki arwiki cawiki huwiki rowiki ukwiki

These will delete more than 1 million rows, and have 90%+ rcs coming from wikidatawiki." Replying to this, admin Reedy said "I think we should just disable it everywhere till Wikidata confirm it's fixed."

So; not a dozen but 15 other language versions, including the three I mentioned; or all of them, depending on which opinion prevails in the end. Bck to the issue at hand: "If you want to catch Wikidata vandalism immediately, then you can always use the recent changes page on Wikidata." But that's not what the main Wikidata-supporters are always claiming, isn't it? The idea is that Wikidata can be safely used here, because you can patrol the changes here, on your watchlist and recent changes (though not in page history). To now dismiss these concerns with "but you can watch the changes there as well" is quite a big leap, and a position which will be very unpopular with many editrs who may reluctantly accept Wikidata data (in infoboxes and so on) otherwise. Requiring people to watch two watchlists or two pages with recent changes on different sites, and without any indication on Wikidata Recent Changes whether these items even have an enwiki page, will most likely not fly at all. "For most cases of watching changes to the content of infoboxes etc., then a delay of this kind of length isn't vital" That's not the problem though, isn't it? The delay means that not a single recent changes patroller, even if they have "Wikidata" enabled, will ever see these changes. Basically, this delay completely disables one of the two main checks enwiki has of Wikidata content, and makes it harder to check on the watchlist as well (as you can't just check new changes at the top, but also have to check for suddenly appearing changes from 1/2 hour or longer ago). Whether the changes themselves would be visible in the articles before they would reach the watchlist isn't clear to me: if so, it would make it even worse. I note from your link that enwiki is the slowest to get the changes (presumably because we are the largest?). The lag described at that page doesn't seem to match the end result at enwiki though; it now claims a 3-minute lag (down from a 9-minute lag just before), but in recent changes the most recent Wikidata entry is 6 minutes old, still way off the 500-changes limit (standard is 100 most recent changes, 500 is the most the dropdown provides, 1000 is the most the page happily accepts, and 5000 is the most I can force if necessary). Fram (talk) 13:19, 17 October 2017 (UTC)[reply]

@Fram: Probably the more important point however, Fram, is that in the longer term this issue is ultimately likely to have a highly positive out-run. People have been complaining for at least two years that activating Wikidata changes was impractical because all it did was create a flooded watchlist with changes that were irrelevant to the en-wiki articles they were actually interested in. The current difficulties have (at last) now forced that issue to the top of the Wikidata dev team's agenda.

Already there was a trial fix being piloted on el-wiki, to only show changes in statement-values or descriptions that were actually accessed by the page.

Active study of the present issues is leading to further fixes, eg to limit changes that have very very wide effects; also some coding tweaks to templates and lua modules to reduce statement accesses that are not in fact needed.

It's now a priority to make sure only relevant Wikidata changes are propagated to recent changes. An end to the flooding should make Wikidata-activated watchlists (at last) finally usable. Jheald (talk) 22:32, 17 October 2017 (UTC)[reply]

The more important and positive point is that more people will probably realise that Wikidata isn't a cost-free solution and that not relying on Wikidata is the more robust, intuitive, maintainable and easy solution (easy, not lazy, that is; relying on Wikidata is the lazy solution). Fram (talk) 04:30, 18 October 2017 (UTC)[reply]

Delay is now 1 hour and 23 minutes (according to this, thanks to Mike Peel for showing me that page) and rising. It has been less than 2 minutes, but ha been steadily increasing again. Other wikis have similar issues (I have seen it at 44 minutes for Swedish wiki), but enwiki is most often the slowest. Fram (talk) 11:38, 18 October 2017 (UTC)[reply]

It has gone up to about 2 hours and 20 minutes; but is now slowly decreasing again (currently at 2 hours 15 minutes). So this wasn't just a spike or glitch, but a persistent problem. At the moment more than 100,000 changes have not been dispatched to enwiki yet. Now, this may not seem like a problem, as it means that vandalism isn't dispatched here either; but a) it won't appear on recent changes and the like, and b) if it gets noted and corrected on Wikidata anyway, the correction will also take hours to get here (just imagine the frustration of someone spotting an error or vandalism coming from Wikidata, correcting it there, and then purging and refreshing fruitlessly for hours on enwiki to get rid of the vandalism and to see his correction; and compare this to the situation with errors and vandalis directly on enwiki). Fram (talk) 14:30, 18 October 2017 (UTC

The decrease swiftly ended, it was still above 2 hours last night and was above 3 hours this morning, having now again dropped minimally to 2 hours and 51 minutes. This is a persistent and serious problem, rendering Wikidata changes totally invisible in recent changes and for all purposes invisible on long watchlists (my watchlist screen is filled with changes from the last 40 minutes, I would have no reason to scroll down to see if between the already checked changes from 3 hours ago suddenly some Wikidata stuff appears. Fram (talk) 08:20, 19 October 2017 (UTC)[reply]

And now up to 4 hours and 19 minutes. Time to have that RfC, me thinks. Fram (talk) 12:39, 19 October 2017 (UTC)[reply]

@Whatamidoing (WMF): I think you'll be interested for this discussion. -- Magioladitis (talk) 16:41, 19 October 2017 (UTC)[reply]

Update: We got the dispatch lag down to a few minutes now as it should be. Seems there was a hickup but we'll look more into it tomorrow. --Lydia Pintscher (WMDE) (talk) 22:21, 19 October 2017 (UTC)[reply]

The one time on Saturday I checked this, it was a lag of more than 1 hour. On Sunday it looked okay again, but this morning it was constantly between 30 minutes and 1 hour for a few hours before it dropped now to 20 minutes. Seems to be a constantly returning problem, not a one-off hiccup. Fram (talk) 11:40, 23 October 2017 (UTC)[reply]

Yesterday, the Wikidata item for the country Romania (not really an obscure topic I would think, one that is used in thousands of other Wikidata items) was moved (english label changed) to Moldavia, with vandalism to the description (!) and alias as well. This was only reverted nearly three hours later[17]. This is pretty quick by Wikidata standards, e.g. yesterday as well Fernando Alonso (another high profile article) was vandal-moved in English, Spanish and Catalan for more than two hours[18], while more obscure articles take about 8 hours to be reverted (labeling a living person a "fascist" seems like a quite obvious case of vandalism[19]), if they get spotted at all (I'm watching another high profile BLP, with articles in over 70 languages and more than 4000 pageviews per day on enwiki alone, which has been vandalized on Wikidata in September and is unlikely to be reverted any time soon probably).

Anyway, back to the issue at hand: the Romania vandalism was reflected in quite a few enwiki articles, including the UNESCO world heritage sites but basically every infobox that fetches information from Wikidata and includes the field "Romania" (biographies, location of artworks, observatories, ...). The end result was similar to what you can see in the images; in these cases, the country name was changed, but also the wrong map was shown, and the red dot location indicator was somewhere in the middle of the page instead of in the infobox. Basically, using Wikidata makes vandalism on many articles much easier, and is almost guaranteed to remain in the articles for a lot longer as well. Coupled with the recent changes delay (so even if you have Wikidata in recent changes enabled, chances are you wouldn't see this happening) and the problem that these don't appear in the page history, and you end up with a situation which is beneficial to vandals and negative for recent changes patrollers. Fram (talk) 07:46, 18 October 2017 (UTC)[reply]

This one is the real problem. High-profile articles on Wikidata can not be protected indefinitely (at least not in large numbers) because this would block small projects (where editors typically are not autoconfirmed on Wikidata) from linking these articles on Wikidata, adding descriptions in their language and moving them. I raised the topic on the Wikidata Project Chat some time ago (I believe in September, difficult to find now), and the only reasonable suggestion was to run a anti-vandal bot, bot nobody volunteered to write the bot, and the topic was eventually archived. I think indeed until the vandalism problem in high-profile articles has been resolved the integration of problems with Wikidata will remain very limited.--Ymblanter (talk) 08:13, 18 October 2017 (UTC)[reply]

Possibly a finer grained protection on Wikidata might be useful. Or a completely different mechanism. All the best: Rich Farmbrough, 19:36, 18 October 2017 (UTC).[reply]

An anti-vandalism bot is running and can be extended. It is d:User:Dexbot. --Lydia Pintscher (WMDE) (talk) 13:51, 19 October 2017 (UTC)[reply]

Hmm, first example I looked at, [20], is a helpful, good IP edit (or certainly not the kind of vandalism edit that needs reverting!) being reverted by the bot, while what for humans looks like blatant vandalism remains undetected. This as well seems like an incorrect bot edit. Does Wikidata have a WP:BITE policy? Perhaps it needs one... Fram (talk) 14:10, 19 October 2017 (UTC)[reply]

Seems more like a problem with the ORES algorithm. I've done so on d:Wikidata:ORES/Report mistakes. Jo-Jo Eumerus (talk, contributions) 14:39, 19 October 2017 (UTC)[reply]

Good, tnx Lydia.--Ymblanter (talk) 17:06, 19 October 2017 (UTC)[reply]

Example:

• Chorale, Wikidata Q724473

A problem, causing a lot of fruitless discussion on en.Wikipedia talk pages, is that de:Choral, correctly coupled to, among others, chorale at en.Wikipedia, can mean a lot of other things too in German (including e.g. Gregorian chant) not covered by any meaning of the English-language counterpart; while, on the other hand some English-language meanings of chorale might be translated to another term in German (e.g. de:Geistliches Lied).

For clarity: the meaning of English chorale and German Choral are exactly the same in most cases, e.g. Fantasy and Fugue on the chorale "Ad nos, ad salutarem undam" = Fantasie und Fuge über den Choral "Ad nos, ad salutarem undam" (emphasis added) – just trying to take away any potential doubts that the coupling made by Q724473 wouldn't have been correct.

What can Wikidata do to help sort out such issues of not completely overlapping meanings in different languages? --Francis Schonken (talk) 12:00, 23 October 2017 (UTC)[reply]

As I understand it, you can report such issues at d:Wikidata:Interwiki conflicts. It may then take a long time until they are resolved. —Kusma (t·c) 13:04, 23 October 2017 (UTC)[reply]

Thanks, but no: the thus-labelled interwiki conflicts are all about "how-to-couple?" issues. As I tried to make extensively clear above, there's no residue of a "how-to-couple?" question in the given example: the couplings are correct, they are satisfactory and afaics there is nothing to be resolved there (it would seem almost devious to recast this as an interwiki conflict: there is none). --Francis Schonken (talk) 13:50, 23 October 2017 (UTC)[reply]

One common argument against using Wikidata is that Wikidata short descriptions are less visible to Wikipedia editors, with potentially lower quality and susceptibility to vandalism. To fix this, I propose putting the description in smaller, gray text to the right of the article title, in the desktop UI, with an edit button next to it. Clicking that edit button would provide a form for editing the short description, and perhaps also the title. Editing the short description would edit the Wikidata value, and editing the title would move the page. —{{u|Goldenshimmer}}|✝️|ze/zer|😹|T/C|☮️|John15:12|🍂 02:56, 30 October 2017 (UTC)[reply]

An editor, User:ScorumME, with no edits to Wikidata whatsoever and no information on his edits on other wikis, appears, and proposes/requests a bot as his very first edit[21]. At the bot request, User:Mbch331 (an admin there) requests to make a few test edits. The bot owner complies, and User:Ymblanter, another Wikidata admin and bureaucrat, checks the test edits and approves the bot.

The test edits which lead to the approval of this bot are:

• [22]
• [23]
• [24]

Looking at these additions, we see that all entries are referenced. To [25] https://scorum.me/football/tourneys/1103-2/j-league. Notice anything particular about that website? Right, the website name is the same as that of the bot owner.

It gets worse. When you follow that link, you go to [26] https://scorumcoins.com/, a site to promote a new cryptocurrency with "Tokens Crowdsale Starts In X days": "1 SCR = 1 USD WE ACCEPT ONLY BTC AND ETH" and so on. Scroll down, down, down and you'll find "KEY PARTNER: Microsoft" (may be true or not, no way or interest to verify) and "OUR FRIENDS: Wikipedia":

"we've partnered up with Wikipedia to provide sports fans with the match data from Scorum right on Wikipedia pages." Um, what? No way, José. You've infiltrated one unreliable affiliate of Wikipedia, where even people with serious responsabilities seem to accept no matter what, no questions asked. That doesn't mean that you've partnered up with Wikipedia, or that e.g. enwiki as a whole would ever accept your contributions (some pro-Wikidata editors may accept this shit, but let's hope they are still a minority here).

Does the site even contain the statistics it is supposed to source? Well, I can go to here, but clicking on "Japan" gievs no result. I'm not able to type into the search box either.

This is the kind of site and editor which gets speedily promoted to "bot" so they can edit at high speeds, create numerous pages like the same one five times in a row[27][28][29][30][31] even though they had already created the same page a month earlier[32]. Never mind that Wikidata already had an item for this[33]...

So please tell me again, why should we trust this site, Wikidata for anything at all, if their admins and bureaucrats promote spambots and no one checks their history or contributions either before or after this promotion? Fram (talk) 12:36, 30 October 2017 (UTC)[reply]