Talk:Machine Intelligence Research Institute

NPOV for Pakaran

I've taken a lot of stuff out of the article that seemed to be basically just handwaving and self-promotion. This is what it read like when I found it:

"The Singularity Institute for Artificial Intelligence is a non-profit organization that seeks to create a benevolent artificial intelligence capable of improving its own design. To this end, they have developed the ideas of seed AI and Friendly AI and are currently coordinating efforts to physically implement them. The Singularity Institute was created in the belief that the creation of smarter-than-human, kinder-than-human minds represents a tremendous opportunity to accomplish good. Artificial intelligence was chosen because the Singularity Institute views the neurological modification of human beings as a more difficult and dangerous path to transhuman intelligence."

"The Singularity Institute observes that AI systems would run on hardware that conducts computations at billions or trillions of times the characteristic rate of human neurons, resulting in a corresponding speedup of thinking speed. Transhuman AIs would be capable of developing nanotechnology and using it to accomplish real world goals, including the further enhancement of their own intelligence and the consensual intelligence enhancement of human beings. Given enough intelligence and benevolence, a transhuman AI would be able to solve many age-old human problems on very short timescales."

As it stands, that isn't a bad article, it's just that it isn't really suitable for an encyclopedia. It presents some things as fact that are clearly opinion. It makes contentious statements, such as that it originated concept of "Seed AI" (astonishing for such a new organization--I read similar ideas in Von Neumann's book in the mid-seventies, and that had been written nearly thirty years before). The claim to be "coordinating efforts to physically implement" Seed AI and Friendly AI seem to rest on fundraising and writing a lot of papers about an extremely loosely defined programming language which seems to lack even an experimental implementation.

Wikipedia isn't for original research, it isn't for us to put up our pet ideas (however valid they may be). It's to catalog human knowledge from a neutral point of view. The article as it stood was in my opinion not so much an encyclopedia article as a promotional panegyric. --Minority Report 03:07, 23 Nov 2004 (UTC)

Ok, in the interest of admitting biases, I'm a financial donor to the SIAI. It's true that there have been holdups in beginning actual development, largely because there's a need to get all the theoretical underpinnings of Friendly AI done first.

That said, claiming that the SIAI is a "religion" rather than a group (which you may or may not agree with) is intrinsically PoV. --Pakaran (ark a pan) 03:55, 23 Nov 2004 (UTC)

I agree with most of your criticisms, Minority Report, and the article was not NPOV as it existed before. The statement that they are coordinating efforts to implement seed AI is quite valid, however. SIAI is developing a smaller, less ambitious AI program, although the primary objective of its research now is formalizing the theoretical framework for Friendly AI.

Also, using the phrase "quasi-religious" to describe an institution that claims to be entirely secular is highly misleading. SIAI has no affiliation with any religion.

I'm interested in your comments regarding von Neumann's work. I was not aware that von Neumann had speculated in this area. If you can find a source perhaps it should be mentioned at Seed AI. — Schaefer 05:02, 23 Nov 2004 (UTC)

I think my use of the term "quasi-religious" was an overstatement. I was trying to encapsulate the visionary aspect of this work, and the use of language which seems to owe more to religion than to engineering. I apologise if I also mischaracterized the Seed AI stuff; from looking around the site I saw a lot of hot air and little activity. I read a few books by Von Neumann in the late seventies, and the idea of having self-improving machines was very much his aim. I'm sorry I can't recall the specific book. I thought it might be The Computer and the Brain but a glance at the contents page on Amazon doesn't offer any clues. The idea was certainly in the air in the 1970s, long before Vinge's 1993 paper. --Minority Report 11:09, 23 Nov 2004 (UTC)

I've added basic information on the SIAI-CA and removed an erroneous middle initial. I also changed the first paragraph to reflect the fact that the SIAI actually does want to build software, rather than just talk about it, and to clarify that the 'Singularity' in the name refers to influencing the outcome of a technological singularity. --Michael Wilson

Merges

I have merged in information from the previously separate items on Emerson and Yudkowsky, which amounted to about a line of exposition and a few links. Those items now redirect to this item.

Yeah, I'd like that redirect to be removed. Actually, I'm removing it now. Eliezer Yudkowsky is wikified in many articles already. There is no reason to redirect an article about a person to their association's article. Biographical articles can be fleshed out over time and as of now it *looks* like we don't have an article on Yudkowsky when in fact we did. A line would have been a good start for smeone to write more. I'm making Eliezer Yudkowsky a bio-stub. --JoeHenzi 22:52, 21 Apr 2005 (UTC)

GiveWell and OpenPhil evaluations

As of two days ago, the article ended with this section:

In 2016 GiveWell, a charity assessment organization based on the effective altruism concept, did an extensive review of MIRI's research papers, and concluded that "MIRI has made relatively limited progress on the Agent Foundations research agenda so far, and this research agenda has little potential to decrease potential risks from advanced AI in comparison with other research directions that we would consider supporting".

I added the following section:

Subsequently the Open Philanthropy Project, a charity evaluator spun off from Givewell, evaluated MIRI more positively in 2017. They awarded MIRI a grant for $3.75m, supporting roughly half MIRI's budget. While acknowledging that MIRI was very hard to evaluate, they wrote that "The case for the statement “MIRI’s research has a nontrivial chance of turning out to be extremely valuable (when taking into account how different it is from other research on AI safety)” appears much more robust than it did before".

The second section seems to me to be just as relevant as the first - they both concern evaluations by (essentially) the same organisation, one year apart. Perhaps a slight claim could be made that the second one is slightly more relevant, as it is more recent.

David Gerald reverted my edit, citing primary sourcing. However, if true this would disqualify both paragraphs, as both use basically the same source. I think both contain useful information, so would prefer both to stay, rather than neither. Furthermore, according to USEPRIMARY, "Sometimes, a primary source is even the best possible source, such as when you are supporting a direct quotation." When I reverted him and pointed this out, David reverted my edit for a second time, citing "rv promotional edit". I'm not sure what exactly he means by this. I haven't been paid by MIRI, for example. It's true that the second paragraph paints MIRI in a slightly better light than the first one, but this is because the opinion of the charity assessment organization improved. Presumably it is not the case that adding any positive content at all is forbidden?

As such I am going to re-add the section, though I welcome David's comments. 69.141.26.148 (talk) 15:53, 24 March 2019 (UTC)

The wording seems misleading to me based on the source. The first part of the quote, "While the balance of our technical advisors’ opinions and arguments still leaves us skeptical of the value of MIRI’s research..." is not included. It seems that a note regarding the fact that Open Philanthropy Project funded MIRI is worth including, but the language used above is unnecessarily promotional. The primary source quoted seems to be quite nuanced. The addition either needs to add that nuance, or just make a note of the facts, i.e. they're granting $3.75 million over 3 years. Although a secondary source would be nice, it is understandable that it might be hard to produce. As such, a promotional edit doesn't necessarily mean a COI. For articles like these, it is important to observe WP:NPOV. --Btcgeek (talk) 17:50, 24 March 2019 (UTC)

How about something like the following? I think the sentence flow is a little more awkward now, but it captures the additional section you pointed to. 69.141.26.148 (talk) 01:09, 26 March 2019 (UTC)

Subsequently the Open Philanthropy Project, a charity evaluator spun off from Givewell, evaluated MIRI more positively in 2017. They awarded MIRI a grant for $3.75m, supporting roughly half MIRI's budget, writing that "[w]hile the balance of our technical advisors’ opinions and arguments still leaves us skeptical of the value of MIRI’s research, the case for the statement “MIRI’s research has a nontrivial chance of turning out to be extremely valuable (when taking into account how different it is from other research on AI safety)” appears much more robust than it did before".

I suggest removing the "evaluated MIRI more positively in 2017" part. The source doesn't actually say that. They say they remain skeptical, but there are these 2 reasons why they increased their funding. It would also be nice to note that the grant of $3.75m is over 3 years - as it reads now, it seems like a one-time grant, but it isn't. Here's a slight edit of what David Gerard noted above as my suggestion - --Btcgeek (talk) 01:39, 26 March 2019 (UTC)

Subsequently the Open Philanthropy Project, a charity evaluator spun off from Givewell, awarded MIRI a grant for $3.75m over 3 years, supporting roughly half MIRI's budget, writing that "[w]hile the balance of our technical advisors’ opinions and arguments still leaves us skeptical of the value of MIRI’s research, the case for the statement “MIRI’s research has a nontrivial chance of turning out to be extremely valuable (when taking into account how different it is from other research on AI safety)” appears much more robust than it did before".

Seems not unreasonable to me. I'll add it to the article. Thanks! 69.141.26.148 (talk) 22:59, 28 March 2019 (UTC)

I disagree - you still don't have any reliable third-party sourcing that anyone in the real world has even noticed this. Please find a third-party Reliable Source, not a self-source. Surely this isn't hard, if this is worth even noting here. Else it's just promotional - David Gerard (talk) 06:47, 29 March 2019 (UTC)

David Gerard, I agree that it should have a third-party source. However, the preceding information about the Open Philanthropy Project, which you seem to wish to keep in, is also a primary source. I suggest that either both are included (since they are simply coverage from the same source, a year apart), a combination of the two is kept in, or neither is kept in. Gbear605 (talk) 14:22, 29 March 2019 (UTC)

I would remove both, if there's no evidence of third-party coverage in reliable sources. Is there? - David Gerard (talk) 19:07, 29 March 2019 (UTC)

I do not support its removal. WP:PRIMARY mentions cases where primary sources can be used. There is no requirement in Wikipedia per se that everything needs a secondary source. The use here of primary source seems to have been done carefully and free of interpretation, which generally requires a secondary source. I don't support the entire deletion of material that seemed to add value to the article, giving a context of the work done by this organization. — Preceding unsigned comment added by Btcgeek (talk • contribs) 21:47, 29 March 2019 (UTC)

You're still supplying absolutely no evidence that this is in any way a noteworthy fact, and that putting it here would be anything functionally other than advertising. Let me repeat again: Do you have third-party coverage in Reliable Sources? This is a pretty simple question, with a yes or no answer - David Gerard (talk) 11:23, 31 March 2019 (UTC)

Please read WP:PRIMARY to understand how primary sources are used in Wikipedia, as mentioned to you previously.--Btcgeek (talk) 01:44, 1 April 2019 (UTC)

So that'll be "no, I have no evidence that any third party anywhere has noted this at all"? Then it's just promotional fluff about MIRI - David Gerard (talk) 07:42, 1 April 2019 (UTC)

You're causing disruptive edits to the article without seeking consensus. I've provided you with the right Wikipedia policies to go through already regarding primary sources. The material you deleted was relevant to the article and accurate based on the sources. It didn't show MIRI in the best light, but that's ok and not the purpose of the Wikipedia article. Things like funding sources and independent evaluations are very important for projects that seem "out there" as there is potential for misrepresentation and fluff. Wikipedia tries to provide a holistic view of the organization, not just things that are promotional in nature. You still haven't provided any reason for the removal of this material other than saying you couldn't find secondary sources. It is pretty obvious that the material you removed ins't "promotional fluff" and is in fact the opposite - citing questions about their research and healthy skepticism that you somehow want to keep out of the article. Do you have a COI here? --Btcgeek (talk) 19:55, 1 April 2019 (UTC)

David Gerard is an admin for RationalWiki, which has a one-sided negative view toward MIRI, which can be seen in its pages on Eliezer Yudkowsky and LessWrong. He seems to very much not be a neutral party on this issue. Gbear605 (talk) 20:12, 1 April 2019 (UTC)

This sort of thing is normally considered a violation of WP:AGF and WP:NPA. Rather than attempting personal attacks upon other Wikipedia editors, I urge you to address the sourcing problems. In particular - do you have third-party reliable source coverage? If you can bring such, it will be a slam-dunk argument in favour of keeping the material - David Gerard (talk) 21:51, 1 April 2019 (UTC)

"You're causing disruptive edits to the article without seeking consensus." I am seeking consensus, which is why I'm here. But - and this is important - a talk page can't agree on its own to contradict fundamental sourcing rules of Wikipedia.

The reason for the removal is what I've said more than a few times - it's entirely self-sourced puffery, which appears to have zero third-party reliable sources coverage. I've said this a number of times; your claim "You still haven't provided any reason" is clearly factually false, and it's an assertion you should reread this page before making.

Do you have third-party, verifiable, reliable source coverage for the claim? Yes or no? This is a simple question, and you've yet to answer it. I must note yet again - if you can find such coverage, it will be a slam dunk case for it to be included in the article - David Gerard (talk) 21:51, 1 April 2019 (UTC)

This is what you've deleted from the article in your last deletion: "In 2016 GiveWell, a charity assessment organization based on the effective altruism concept, did an extensive review of MIRI's research papers, and concluded that "MIRI has made relatively limited progress on the Agent Foundations research agenda so far, and this research agenda has little potential to decrease potential risks from advanced AI in comparison with other research directions that we would consider supporting". Please explain how this is "self-sourced puffery". To me, the content seems clearly not self-sourced and clearly not puffery. --Btcgeek (talk) 04:02, 2 April 2019 (UTC)

Citing an organisation about claims the organisation is making is primary sourcing. That's what the term means.

Do you have third-party, verifiable, reliable source coverage for the claim? Yes or no? This is a simple question, and you've yet to answer it. - David Gerard (talk) 07:30, 2 April 2019 (UTC)

Ok, now we're getting somewhere. First, confirm that "puffery" is no longer your concern, and that the material should not have been removed for reasons of puffery/self-promotion. Second, the sources that you removed seemed to follow all the guidelines of third-party verifiable and reliable sources that Wikipedia needs. The source was about GiveWell/Open Philanthropy writing about MIRI, and not MIRI writing about MIRI, so clearly it isn't a first party source. The material from the source is easily verifiable. GiveWell/Open Philanthropy is a fairly reliable source in my opinion. Now that we've broken down your concerns, which of these is exactly your concern?

1. An article about MIRI citing a GiveWell/Open Philanthropy source is primary source because you believe this is PR material put out by MIRI and not actually attributable to GiveWell/Open Philanthropy.

2. You're unable to verify that the quote that was removed from the article cannot be verified to belong to the source.

3. You believe GiveWell isn't a reliable source on Wikipedia and should be added to unreliable sources list on Wikipedia (provide your reasoning if this is the case).

The way the source works in this instance is similar to how we use sources in Wikipedia say to cite the Alexa rank of a website, or ranking lists from sources (e.g. list of countries by GDP say). Let's try to resolve this by laying out your concerns and addressing them specifically. --Btcgeek (talk) 14:41, 2 April 2019 (UTC)

You're still evading the question! Do you have third-party, verifiable, reliable source coverage for the claim? Yes or no? This is a simple question. Please answer it before you throw out other questions that aren't this question - David Gerard (talk) 20:00, 2 April 2019 (UTC)

Why are you asking the same question so many times without reading my replies? To reiterate, the source you deleted is a. third-party, verifiable, reliable source. First, you claimed the source was puffery. Then you implicitly backed off that claim which you clearly knew was not true after I pointed this out to you. Then you claimed it's somehow first-party when I have clearly told you several times now that it's not a primary source since the source is from GiveWell/Open Philanthropy and the article is about MIRI. Then you're somehow claiming this isn't verifiable and/or not a reliable source for Wikipedia. Before we go any further please answer my questions above without further evasion. If you think the source that you deleted is a primary source, for example, explain how a GiveWell/Open Philanthropy source is a primary source for an article on MIRI. --Btcgeek (talk) 21:46, 2 April 2019 (UTC)

You're citing the claim to one of the organisations directly involved in the claim - that's a primary source. That's what the words mean. It is not a Wikipedia-meaning Reliable Source - an independent third-party source on the claim. You're making completely incorrect claims about the quality of your source - David Gerard (talk) 21:38, 3 April 2019 (UTC)

I am withdrawing from this conversation. It doesn't seem like you're willing to do the right thing even when I've given you all the evidence and examples, and you continue to grossly misrepresent the sources that you deleted from the article. --Btcgeek (talk) 22:50, 3 April 2019 (UTC)

And if the fact hasn't been noted anywhere, it's not notable. Has any third-party RS even mentioned it? Surely this is not a high bar - David Gerard (talk) 20:05, 24 March 2019 (UTC)

The notability guidelines apply to whole articles, not contents of articles (see WP:NNC), and it is of obvious interest how an organisation is financed. The question here should be reliability of sources. — Charles Stewart (talk) 17:17, 31 March 2019 (UTC)

It is worth noting that (i) MIRI publishes audited accounts, and (ii) that since potential donors might be swayed by the fact of and reasoning behind the donation, claiming the donation took place when it did not would be fraudulent. We could simply ask the auditor to confirm that the donation took place. — Charles Stewart (talk) 12:09, 3 April 2019 (UTC)

References

New Open Phil grant

User:David Gerard, isn't WP:PRIMARY acceptable in this case? It can be supplemented with Open Phil's page on the grant, as well as an additional non-primary source once available. - Indefensible (talk) 16:56, 28 April 2020 (UTC)

Unless anyone else in the world cares about it, it's likely WP:UNDUE, I'd think - or the "yes, but so what?" test. I mean, others might disagree with me, sure. But nonprofits whose WP pages are encrusted with their own blog posts about themselves, detailing things that no third parties bothered documenting, tend to be the ones that really shouldn't have that much detail in their articles - David Gerard (talk) 19:52, 28 April 2020 (UTC)

Supplemented with Open Phil ref, at minimum this meets the existing 2019 grant sentence. - Indefensible (talk) 05:02, 1 May 2020 (UTC)

While it is unfortunately common to list grants in some articles, this still should be explained using neutral language. Ideally with context from better sources. Grayfell (talk) 05:40, 1 May 2020 (UTC)

Eliezer has left the building

Do we have WP:RS for this yet? He mentioned it on this podcast. - Scarpy (talk) 12:12, 24 February 2023 (UTC)