In January 2017, the Signpost polled its readers. We sought to learn more about our readers' habits and wishes, around subscription and notifications. We were also interested in the dynamics that bring readers to us in the first place; we believed that readers typically learn about the Signpost by finding it on their colleagues' user talk pages, but we wanted to test that hypothesis.

The poll was prompted by recent progress on a long-planned extension to Wikipedia's underlying software, which will offer a new, central page on which publications may advertise their existence, and will allow publishers to notify their readers of new issues or editions via web or email notifications instead of user talk page messages.

We also have an important (but only tangentially related) development to report. Thanks to the efforts of Evad37 and Samwilson, the Signpost once again has a functional RSS feed, here. The feed is still being refined, but is usable as of now.

Analysis

Between January 17 and February 2, 2017, we received 93 responses.

• 54 respondents supplied usernames.
• 61 identified themselves as subscribers; 30 said they were not, and two gave no answer to that question. (Note, we did not supply a specific definition of "subscriber.")
• 74 listed the English Wikipedia as their "home wiki." 3 Chinese Wikipedia, 2 each from Wikidata and the French and Spanish Wikipedias, and 1 each from English Wikiquote, MediaWiki, and from the Danish, Dutch, Galician, German, Hungarian, Norwegian, and Swedish Wikipedias.

In the near term, these data will inform our decisions about the Newsletter Extension. Though it is outside the scope of our decision and our sample, the results may prove helpful to the English Wikipedia more broadly, if and when it makes a determination about whether and how to implement the extension. We were pleased with the level of response, and may run similar polls in the Signpost in the future.

How did you first learn about the Signpost?

Wikipedia users often learn from each another through interactions on user talk pages. When one Wikipedian sees the Signpost on a colleague's user talk page, that may be roughly analogous to noticing a magazine sitting on their table; when a Signpost notification appears in the watchlist, that may be similar to seeing a newspaper on a friend's doorstep.

The Signpost enjoys strong readership and community engagement; but that's not something we can take for granted. We therefore wanted to learn more about how our readers originally learned of the Signpost. 40% of poll respondents learned from a user talk page, which would not be part of a system based on the new extension. Another 40% learned of the Signpost on a wiki; while the extension would offer an on-wiki list of newsletters, there is no easy way to predict behavior patterns around visiting a page that doesn't yet exist. Only with extensive research (beyond the remit of either the Signpost team or the Newsletter Extension team) might we develop a strong theory about how Wikipedians might become aware of the Signpost (or other newsletters) if the delivery methods were to change substantially.

We interpret this as a strong reason to approach with great caution any changes to delivery that would eliminate user talk page notification.

How do you usually find a new edition of the Signpost?

We were curious about what attracts our readers' attention when we publish a new edition. Since our main subscription options involve delivery to user talk pages and updated information in user page templates, we were not surprised to see more than half of respondents are alerted within their own user space.

One noteworthy result is that 7.6% learn that we have published a new edition via somebody else's user space, echoing the results of the previous question. Interestingly, three of the seven respondents who gave this answer also described themselves as subscribers. We would not have expected this as the primary answer from readers who identify as subscribers. This suggests that to some of our readers, the appearance of the Signpost in a familiar place may be part of the process that draws them into our pages, in addition to the formal notification that results from subscription.

What's your preferred way to receive the Signpost?

55% of respondents prefer notification on their user talk pages, as currently offered. While it's important to consider the role of selection bias, this is an especially strong result, and one we cannot afford to ignore. If even a few of our readers (say, 10%) preferred to have user talk notification, it would be difficult for us to justify doing away with it; but this goes far beyond a significant minority. Defying the preference of a majority is not a reasonable option, meaning that the Signpost cannot consider eliminating the present delivery method for the foreseeable future.

It was interesting to learn that 16% of our readers would prefer to receive the Signpost by email. This leaves an open question; since we do send notifications to two email lists (WikimediaAnnounce-L and Wikimedia-L), we don't know without further inquiry how well we are meeting the demand for email notification. If readers would prefer a direct email notification apart from those lists, that is something we may wish to consider in the future.

Our subscriber list has always been publicly visible. Should this continue?

Throughout most, if not all, of the Signpost's history, we've maintained publicly visible subscriber lists. (Those wishing to subscribe privately do have alternatives, however, such as subscribing to one of the email lists noted above, adding the Signpost issue page to their watchlist, etc.)

While we've heard no complaints about subscription privacy, we did learn that keeping subscriptions private was a goal of the extension's design team, so we included this question. We also considered that, while publications have historically used subscription methods that are at least somewhat private, many modern digital publications (such as Medium, Facebook, and Twitter) treat public expressions of interest and affiliation as a feature, not a bug.

Only 5.6% of respondents preferred that the subscription lists be kept private. We hope our current menu of options (including publication to two email lists) is adequate for those readers, but can't be certain without further inquiry.

How important is it to see our story titles as links on your talkpage?

58.7% prefer to have the titles and links to each section visible in their notifications.

Conclusions

The primary purpose of this poll was to inform the Signpost's plans: should we anticipate transitioning to the new, Echo-based Newsletter Extension if and when it becomes available on English Wikipedia? If so, should we do so at the earliest opportunity, or wait? Should we make a clean switch, or use both the old and new methods during a transition period?

Based on our analysis of the results, we do not plan to use the Newsletter Extension in the foreseeable future. We do not see evidence that our readers have a significant problem in need of a solution (nor do we have a significant problem publishing under the current system).

We also feel that the risk of disrupting the notification patterns, as well as the risk of disrupting the dynamics that lead new Wikipedians to encounter the Signpost in the course of their normal editing process, outweigh any potential benefits. Some specific concerns:

1. Confusion for readers, or potential readers, if they encounter a page that purports to offer a comprehensive list of newsletters, and of ways to subscribe to them, but leaves out some newsletters and methods;
2. It would tax our limited volunteer resources to take this on, to get it right, and to maintain an additional notification method during a transition period;
3. Our exposure to new readers could suffer, and we have yet to see a sophisticated theory for how new readers could be exposed to the Signpost under the new system.

While our poll made no effort to reach beyond readers of the Signpost, in the absence of information about broader communities (like all of English Wikipedia, or all of the Wikimedia projects), we feel this poll may be useful to the extension's development team, and may also inform wiki projects' decisions about when, whether, and how to adopt the extension.

On these broader decisions, one point stands out: the Newsletter Extension relies on listing newsletters on a single, central page. If a wiki adopts the extension (at least, as it's currently designed), any newsletters that decline to opt into the new system will not be represented on that central page. This could have the undesirable result of increasing confusion about what newsletters exist, rather than decreasing it.

Regardless of whether and how it is adopted, we applaud the effort to develop new technical tools for MediaWiki users, and appreciate the opportunity to evaluate it for our needs.

Errata

Our pie charts, and their underlying data, simplify the responses to some degree; we changed the wording of some responses to establish clearer patterns (e.g., changing "es.wiki" to "Spanish Wikipedia" so the two would be grouped together under the "home wiki" question, and combining "meta change list" with "elsewhere on Wikipedia", renaming the result to "elsewhere on wiki", for the "How did you first learn about the Signpost" question.) For transparency, each pie chart's description page on Wikimedia Commons links to both the underlying data, and to the more granular pie chart with answers exactly as provided (but with usernames redacted for privacy).

For a complete list of the original poll questions, as well as a chart of the pros and cons of various delivery methods, see here.