Background

There's a browser extension called Shinigami Eyes, relatively popular among trans people and their allies, which marks links as either trans-friendly (green) or transphobic (red) using a non-reversible dataset distributed inside the extension.

Users can report links using the extension, to be marked as transphobic, trans-friendly, or to be cleared. This sends a chunk of the page surrounding a link in the report, and an installation ID which was randomly generated when the extension was first run. These reports are compiled, manually reviewed (or so they say), and eventually a new version of the extension is released with the refined data.

Technical background

(click to expand)

The classification is done by testing normalised URLs against two bloom filters, probabilistic data structures which can confirm membership with a certain degree of confidence, but which cannot list the original data, one for trans-friendly, one for transphobic classifications. If both tests are negative, the URL is left unmarked. Normalisation looks like this:

  • https://twitter.com/AccountName --> twitter.com/accountname
  • https://en.wikipedia.org/wiki/Dag_Hammarskjöld --> wikipedia.org/dag_hammarskjöld

Bloom filters are fundamentally based on comparing hashes, and while collisions are an inherent feature in hashing, the mechanics of a bloom filter, particularly as used here, introduce collisions at a far higher rate. It doesn't really help that the hashing algorithm in question is the non-cryptographic FNV.

This could be exploited in a number of ways, not only via bruteforcing as described below, we've identified a few sets of URLs which, if all marked as trans-positive, would cause prominent people in the "transphobic" bloom filter to be marked as trans-friendly thanks to collisions.

It's also noteworthy that the bloom filter has kept a constant size throughout most releases (last size increase was in 1017), which means that the collision rate is going to constantly increase across releases until such a point as the author adjusts the size

User-submitted reports can be sent using the extension to mark a link as trans-friendly, transphobic, or to clear classification. Reports are encrypted using asymmetric cryptography, and contain a fragment of the page surrounding a link (for example, on Twitter, if you right click on someone's username in a post to submit a report, the report would include that post), as well as an installation ID which is randomly generated and stored when the extension is first used.

The use of the extension to submit the post itself raises questions, it implies that to some extent the manual review process is based on something that could be easily falsified, either by modifying the extension, through the 'inspect element' feature in modern web browsers, or by writing a reimplementation. It's also unclear how these posts are viewed, in the worst case imaginable, an attacker could submit a report containing malicious JS which might be able to exfiltrate the contents of other reports. This is speculation of course, but no answers are forthcoming from the author.

The problems

The main trouble with this project is its lack of transparency and lack of meaningful accountability. The authors and list maintainer are not public knowledge, which makes it doubtful there'd be meaningful social consequences for malfeasance.

Using a bloom filter makes it hard to verify whether any given person was deliberately included or is the result of a false positive, which is exacerbated by the author misusing bloom filters in a way which means that the already unacceptably high risk of false positives increases with every release, and that false positives can persist in the data long-term. This design might as well be optimised for avoiding accountability.

Because bloom filters have predictable false positives, it's practical to derive an unused username, use it, and deliberately assume either trans-friendly or transphobic status. David Buchanan first demonstrated an example of a double false positive, an unused Twitter handle which is deemed both trans-friendly and transphobic simultaneously (@x0s1jpnq2sk2). It transpires that bruteforcing is extremely practical, any competent programmer can generate thousands of unused URLs which are classified as either trans-friendly or transphobic. There's a link below to a longer list of examples.

False or malicious classifications could give people a false sense of security, or cause people to be shunned without good reason.

The inclusion criteria for the trans-friendly list is quite loose, politicians have been added for token support, gimmick accounts on twitter have been added for no clear reason, and there's concern that trans people may be overrepresented, and that it might risk outing people, or being used as a tool to help direct harassment at trans people and supporters. There is no way to request exclusion from the trans-friendly list, although this has been requested on several occasions.

Many of these concerns have already been raised by people in issues on the github project, which are then promptly and studiously ignored by the unknown author. There's 45 outstanding issues, here's anchor links to every single comment the pseudonymous author has ever written:

  • 6 (2021-04-07)
  • 5 (2021-07-28)
  • 4 (2019-07-28)
  • 3 (2019-01-20)
  • 2 (2019-01-20)
  • 1 (2019-01-20)

I've submitted my own issue on the project's issue tracker on 2021-09-07 asking that the author address at least the false positive problem. No response as of yet. (here)

The Norwegian data protection authority (Datatilsynet) issued an order to Shinigami Eyes in 2021 to provide information concerning the legitimacy of the data processing and on what measures had been taken to safeguard data subjects' rights.

Datatilsynet article på norsk, in English

Order itself (in English)

Datatilsynet received no resonse, and on 2021-12-21 announced its intention to ban Shinigami Eyes in Norway, with the final deadline for the author to respond being on 2022-01-17.

Datatilsynet article på norsk, in English

Advance notice of ban (in English)

What can be done?

If you're using the extension, and find that it offers useful functionality to you, I don't want you to take away from this that there's only a binary choice here - to use it or not to.

Many users report that they do not uncritically believe the indications provided, and only use it as a guide to inform further scrutiny, and this is what I recommend; if your final decision on how you engage with someone is truly your own, and not dicatated by SE, then there was no harm. If you recommend the extension to others, you should be careful to inform them that it is not entirely reliable.

If you are an extension user, I similarly recommend that you do not report people as trans-friendly without their consent; a person may not be comfortable with being added to such a list, there's no way for them to request exclusion, it could make them easier to target, and trans people appear to be over-represented on that list. Don't help to maintain a list of trans people.

Many users have found that the extension fills a niche for which there is no replacement. It's not hard to imagine better than Shinigami Eyes, but the most effective harm-reduction must necessarily be something which fills this niche in reality. The problems and strengths with Shinigami Eyes form a sort of specification - a replacement should protect privacy, should be accountable and transparent, should mitigate power structure problems, and should be easy to use.

In the meantime, we can assume that the author of Shinigami Eyes will remain uncooperative, so one possible avenue of harm mitigation could be to fork the extension into a form which is clearer to users about its falliability, which cannot be used to report users as trans-friendly, and to copy only the "transphobic" bloom filter.

I'd also like to briefly address those few who have told me that it's not right to criticise this, as it fills a purpose, and there is no direct replacement. I don't think that this is a good way to view technology, or to view the world. There's a great deal that can be done to make the world fairer and better and none of it will be done if we all convince ourselves that it's not possible.

This project

Explanation on (defunct) Twitter app

(click to expand)

This twitter-linked webapp is based on a reimplementation of the bloom filter algorithm used by Shinigami Eyes, and can tell you the classification of users you follow on Twitter. I've chosen to limit this to the users you follow to limit some of the more harmful effects of the extension itself, because you are hopefully already familiar with their beliefs, and because this is information you could obtain over time by using the browser extension itself. My objective is to help you understand how this view of the world maps onto your own.

Please be careful about how you interpret the results this gives you. The algorithm at the heart of Shinigami Eyes can return false positives, and someone's inclusion in either bloom filter may not reflect their actual beliefs. While I've tried as far as possible to ensure that my reimplementation accurately reflects the behaviour of the browser extension, I also cannot exclude the possibility that there may be differences that I have not been able to identify.

This application is no longer whitelisted, at present anyone can use it. This may be subject to change depending on load and usage. If this happens, please don't contact me requesting addition to the whitelist.

This application requires read-only API access. Read-only does not mean "consequence free" however, enabling read-only apps still risks exposing information you may not wish to be public, for example, private lists you have, who you block, protected account following status, and the timelines of protected accounts you follow. You should consider whether you trust me before authenticating with this app.

This app only uses the API to determine the usernames of accounts you follow. This app does not retain data. Your API access tokens are erased once used, everything else is held purely in memory and not written to disk or logged, with the exception of web server access logs, which will feature your IP address, requested URLs, and timestamps, but cannot otherwise identify you.

Twitter API rate limiting means that this cannot fetch any more than 3000 follows at once. If it shows you a number significantly lower than your following count or 3000 (whichever is lower), please try again after 15 minutes have elapsed.

This software is not currently open-source.

This is currently using the bloom filter data from version 1.0.30 of the Firefox release (released on 2021-08-19)

Contact

For enquiries, please contact me:

Acknowledgements

The final form of this project wouldn't have been possible without contributions from

Datasets

Collisions

These are lists of twitter usernames which would be marked as "transphobic" by shinigami eyes. This is similar to how hash cracking can ordinarily be done, I've written a program which generates usernames and then tests against the "transphobic" bloom filter.

I've decided to only provide lists of "transphobic" bloom filter collisions, because there's less nefarious potential in pretending to be in the "transphobic" list.

Some proportion of accounts in this list are coincidentally real accounts generated by chance, which may or may not have been added to Shinigami Eyes deliberately. Same disclaimer as in the "This project" section.

  • first - First names, largely common British female, urbit sigils, version 1.0.30

Wikipedia articles

(click to expand)

Shinigami Eyes supports Wikipedia articles, and Wikipedia publishes extensive metadata for public use, which presents a unique possibility. It's possible and practical to test every single article on Wikipedia, unlike on the commercial platforms it supports.

Note that these are the normalised versions of the URLs, which differ slightly from the URL you need to access a given article. In particular, instead of wikipedia.org, these should start en.wikipedia.org, and many will have the incorrect capitalisation (Wikipedia is case sensitive, but SE compares in lower-case). In addition, some pages listed here are redirect pages, and it's possible that there may be pages from editions of Wikipedia other than English, SE does not properly differentiate these.

The fact that Wikipedia is an encyclopedia and covers many things which are not people makes it very easy to notice some misclassifications, which are included below. It's impossible to tell if these were added deliberately, added by mistake, or if they're bloom filter false-positives, but some are quite funny.

Version 1.0.30

These two are based on Wikipedia's article name dataset, published on 2021-09-21. This includes every article name on English wikipedia at that moment.

Version 1.0.30 misclassifications

"Trans-friendly"

"transphobic"

Both (these are collisions, which means one or both classifications are incorrect. They'll show as trans-friendly)

Websites

(click to expand) Shinigami Eyes can classify entire websites as trans-friendly or transphobic, so I've run Majestic's million sites dataset through SE. Dataset retrieved on 2021-10-05.