<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: Funding safe AGI</title>
	<atom:link href="http://www.vetta.org/2009/08/funding-safe-agi/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.vetta.org/2009/08/funding-safe-agi/</link>
	<description></description>
	<lastBuildDate>Wed, 01 Feb 2012 19:02:52 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
	<item>
		<title>By: Tim Tyler</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-20550</link>
		<dc:creator>Tim Tyler</dc:creator>
		<pubDate>Sun, 12 Sep 2010 18:34:29 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-20550</guid>
		<description>The closing comments bring to mind the http://en.wikipedia.org/wiki/Vice_Fund</description>
		<content:encoded><![CDATA[<p>The closing comments bring to mind the <a href="http://en.wikipedia.org/wiki/Vice_Fund" rel="nofollow">http://en.wikipedia.org/wiki/Vice_Fund</a></p>
]]></content:encoded>
	</item>
	<item>
		<title>By: jsalvati</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19970</link>
		<dc:creator>jsalvati</dc:creator>
		<pubDate>Tue, 15 Sep 2009 00:47:01 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19970</guid>
		<description>You seem to be somewhat confused (though less confused than many others). The whole idea of Friendly AGI is that &quot;Friendliness cannot be ENFORCED upon any third party&quot; and can only come from shared values, and therefore an important part of AI research must be to find out exactly what the human values are that we want the AI to have (this sounds much simpler than it is).</description>
		<content:encoded><![CDATA[<p>You seem to be somewhat confused (though less confused than many others). The whole idea of Friendly AGI is that &#8220;Friendliness cannot be ENFORCED upon any third party&#8221; and can only come from shared values, and therefore an important part of AI research must be to find out exactly what the human values are that we want the AI to have (this sounds much simpler than it is).</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Bayesian Investor Blog &#187; Blog Archive &#187; Assorted Links</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19969</link>
		<dc:creator>Bayesian Investor Blog &#187; Blog Archive &#187; Assorted Links</dc:creator>
		<pubDate>Mon, 14 Sep 2009 18:18:23 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19969</guid>
		<description>[...] Funding Safe AI (HT Michael Anissimov [...]</description>
		<content:encoded><![CDATA[<p>[...] Funding Safe AI (HT Michael Anissimov [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Shane Legg</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19949</link>
		<dc:creator>Shane Legg</dc:creator>
		<pubDate>Tue, 18 Aug 2009 14:19:28 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19949</guid>
		<description>You&#039;ve lost me regarding smart people and nerds and motivations for behaviours.  My comments on VCs are not based on anybody I know, just talks I&#039;ve watched online where VCs themselves describe their business.</description>
		<content:encoded><![CDATA[<p>You&#8217;ve lost me regarding smart people and nerds and motivations for behaviours.  My comments on VCs are not based on anybody I know, just talks I&#8217;ve watched online where VCs themselves describe their business.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: michael vassar</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19943</link>
		<dc:creator>michael vassar</dc:creator>
		<pubDate>Mon, 10 Aug 2009 20:09:43 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19943</guid>
		<description>Re: VCs, I suspect that you just don&#039;t know smart non-nerds and thus assume smart people have the motivations that smart nerds would have for the same behaviors.  It is a cliche meme.  VERY cliche.

As noted, we need to discuss history of science and technology.</description>
		<content:encoded><![CDATA[<p>Re: VCs, I suspect that you just don&#8217;t know smart non-nerds and thus assume smart people have the motivations that smart nerds would have for the same behaviors.  It is a cliche meme.  VERY cliche.</p>
<p>As noted, we need to discuss history of science and technology.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Shane Legg</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19942</link>
		<dc:creator>Shane Legg</dc:creator>
		<pubDate>Mon, 10 Aug 2009 08:15:48 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19942</guid>
		<description>I think the &quot;gradient of progress&quot; will play a big role here.  If we go from very few people thinking powerful AGI is a near term possibility (i.e. like now) to having a working powerful AGI a couple of years later, then there will be very few teams.  But if most people start to believe it&#039;s a real near term possibility but it takes another 10 years before it happens with a number of increasingly impressive steps being taken along the way, then there could be a large number of groups working on this by the end.</description>
		<content:encoded><![CDATA[<p>I think the &#8220;gradient of progress&#8221; will play a big role here.  If we go from very few people thinking powerful AGI is a near term possibility (i.e. like now) to having a working powerful AGI a couple of years later, then there will be very few teams.  But if most people start to believe it&#8217;s a real near term possibility but it takes another 10 years before it happens with a number of increasingly impressive steps being taken along the way, then there could be a large number of groups working on this by the end.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Shane Legg</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19941</link>
		<dc:creator>Shane Legg</dc:creator>
		<pubDate>Mon, 10 Aug 2009 08:06:50 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19941</guid>
		<description>The guys I saw in this hour long panel discussion about VCs (several years ago... no idea what the link is now) were wealthy and owned successful VCs.  They were not doing badly on average.  Most of their early stage investments failed, but the successes had such massive returns that they made plenty of money.  If this is a cliche meme, it seems odd to me that a group of VC owners would all be pushing this meme on a panel discussion.</description>
		<content:encoded><![CDATA[<p>The guys I saw in this hour long panel discussion about VCs (several years ago&#8230; no idea what the link is now) were wealthy and owned successful VCs.  They were not doing badly on average.  Most of their early stage investments failed, but the successes had such massive returns that they made plenty of money.  If this is a cliche meme, it seems odd to me that a group of VC owners would all be pushing this meme on a panel discussion.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: michael vassar</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19940</link>
		<dc:creator>michael vassar</dc:creator>
		<pubDate>Mon, 10 Aug 2009 04:12:31 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19940</guid>
		<description>Regard AGI, my guess is that if it is developed via brain emulation or very brain inspired appraoches there will be  dozens of serious efforts, each very large and very expensive.  This might also be the case with evolutionary approaches, though I doubt it.  If AGI comes from some other approach I don&#039;t expect it to come from many different sources.  More like two or three groups of a dozen people.   We should probably discuss our different models of scientific progress before or after the Summit.</description>
		<content:encoded><![CDATA[<p>Regard AGI, my guess is that if it is developed via brain emulation or very brain inspired appraoches there will be  dozens of serious efforts, each very large and very expensive.  This might also be the case with evolutionary approaches, though I doubt it.  If AGI comes from some other approach I don&#8217;t expect it to come from many different sources.  More like two or three groups of a dozen people.   We should probably discuss our different models of scientific progress before or after the Summit.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: michael vassar</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19939</link>
		<dc:creator>michael vassar</dc:creator>
		<pubDate>Mon, 10 Aug 2009 03:36:05 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19939</guid>
		<description>OK Shane, but I have a fair amount of experience with VCs, and repetition of that cliche is a meme, not an anticipation controlling belief.  Typical VCs do badly (for reasons I explained grounded in economics) but they don&#039;t anticipate any given company being likely to fail.</description>
		<content:encoded><![CDATA[<p>OK Shane, but I have a fair amount of experience with VCs, and repetition of that cliche is a meme, not an anticipation controlling belief.  Typical VCs do badly (for reasons I explained grounded in economics) but they don&#8217;t anticipate any given company being likely to fail.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: asdf</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19938</link>
		<dc:creator>asdf</dc:creator>
		<pubDate>Mon, 10 Aug 2009 01:22:03 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19938</guid>
		<description>I think individual teams can and will be safe in their endeavors, probably indeed encouraged by organizations like SIAI. I remain worried about government sponsored, in-the-dark projects headed by political figures who don&#039;t understand the risks properly rather than scientists who do understand the grave dangers here.</description>
		<content:encoded><![CDATA[<p>I think individual teams can and will be safe in their endeavors, probably indeed encouraged by organizations like SIAI. I remain worried about government sponsored, in-the-dark projects headed by political figures who don&#8217;t understand the risks properly rather than scientists who do understand the grave dangers here.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Shane Legg</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19937</link>
		<dc:creator>Shane Legg</dc:creator>
		<pubDate>Sun, 09 Aug 2009 21:44:49 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19937</guid>
		<description>Hi Michael,

Regarding VCs, I haven&#039;t dealt with them personally but in videos online where well known tech VC people talk about the business I&#039;ve heard them say that they look for something like 20x returns because the clear majority of companies they invest in don&#039;t make it through to profitability.  I guess that picking the winning AGI team or teams more than a year in advance will be at least as hard and probably harder.  Indeed, even if a safety orientated group such as SIAI supported 10 different promising AGI efforts, my feeling is that there would still be a significant chance of some other group getting there first.  Once real AGI starts to look like a serious near term prospect to many people, there could even be thousands of groups trying from all over the world.

As for the source of investment funds, I&#039;m thinking that it would come more from relatively well off people who were interested and passionate about this kind of thing.</description>
		<content:encoded><![CDATA[<p>Hi Michael,</p>
<p>Regarding VCs, I haven&#8217;t dealt with them personally but in videos online where well known tech VC people talk about the business I&#8217;ve heard them say that they look for something like 20x returns because the clear majority of companies they invest in don&#8217;t make it through to profitability.  I guess that picking the winning AGI team or teams more than a year in advance will be at least as hard and probably harder.  Indeed, even if a safety orientated group such as SIAI supported 10 different promising AGI efforts, my feeling is that there would still be a significant chance of some other group getting there first.  Once real AGI starts to look like a serious near term prospect to many people, there could even be thousands of groups trying from all over the world.</p>
<p>As for the source of investment funds, I&#8217;m thinking that it would come more from relatively well off people who were interested and passionate about this kind of thing.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: michael vassar</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19936</link>
		<dc:creator>michael vassar</dc:creator>
		<pubDate>Sat, 08 Aug 2009 17:17:07 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19936</guid>
		<description>Strategically, I agree with you about SIAI&#039;s main role being the promotion of Friendlyness among other AGI projects.  I think that it may be realistic to expect that our case can be made clearly enough that most competently run AGI projects will eventually be as safety conscious as we think is necessary.  The trend towards greater safety consciousness among younger AGI developers is very strong.  With luck, intelligence enhancement will accelerate this trend still further.   

As a modestly successful entrepreneur (www.sirgroovy.com), I  disagree with you about the superiority of an investment model for raising funds.   Venture and angel funding, as opposed to FFFF (Friends, family and fools funding) is almost exclusively limited to projects that either grossly misrepresent science (e.g. blacklight technologies) or promise a high probability of real results in a reasonable time-frame.  Behavioral economics and my business experiences both agree that humans are not well suited for distinguishing probabilities between .1% and 10%, while any worth-while AGI venture will have to be in the top end of this range.  Ben Goertzel also has a very good record of raising funds, but this is rare in AGI.  He also has a very good record of identifying AI talent that was later recognized elsewhere by those with much greater resources. 

The very best VCs don&#039;t pick mostly duds.  Hummer Winblad, for instance, gets a slight majority of its seed stage software start-ups to acquisition or IPO.  Of course, they are very selective and only fund about 6 companies per year, as well as helping their companies a fair bit. 

Economics dictates that most VCs pick mostly duds, as if VC was in general extremely profitable that would draw money in until most VCs were either unskilled clods or over funded and had no way to effectively use most of their money.

Kevembuangga is amusing.  He calls SIAI idiotic and then immediately makes our chief assertion as if it&#039;s his own idea.</description>
		<content:encoded><![CDATA[<p>Strategically, I agree with you about SIAI&#8217;s main role being the promotion of Friendlyness among other AGI projects.  I think that it may be realistic to expect that our case can be made clearly enough that most competently run AGI projects will eventually be as safety conscious as we think is necessary.  The trend towards greater safety consciousness among younger AGI developers is very strong.  With luck, intelligence enhancement will accelerate this trend still further.   </p>
<p>As a modestly successful entrepreneur (www.sirgroovy.com), I  disagree with you about the superiority of an investment model for raising funds.   Venture and angel funding, as opposed to FFFF (Friends, family and fools funding) is almost exclusively limited to projects that either grossly misrepresent science (e.g. blacklight technologies) or promise a high probability of real results in a reasonable time-frame.  Behavioral economics and my business experiences both agree that humans are not well suited for distinguishing probabilities between .1% and 10%, while any worth-while AGI venture will have to be in the top end of this range.  Ben Goertzel also has a very good record of raising funds, but this is rare in AGI.  He also has a very good record of identifying AI talent that was later recognized elsewhere by those with much greater resources. </p>
<p>The very best VCs don&#8217;t pick mostly duds.  Hummer Winblad, for instance, gets a slight majority of its seed stage software start-ups to acquisition or IPO.  Of course, they are very selective and only fund about 6 companies per year, as well as helping their companies a fair bit. </p>
<p>Economics dictates that most VCs pick mostly duds, as if VC was in general extremely profitable that would draw money in until most VCs were either unskilled clods or over funded and had no way to effectively use most of their money.</p>
<p>Kevembuangga is amusing.  He calls SIAI idiotic and then immediately makes our chief assertion as if it&#8217;s his own idea.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Accelerating Future &#187; Shane Legg on Funding Safe AI</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19935</link>
		<dc:creator>Accelerating Future &#187; Shane Legg on Funding Safe AI</dc:creator>
		<pubDate>Sat, 08 Aug 2009 13:18:45 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19935</guid>
		<description>[...] AI researcher Shane Legg has some interesting thoughts on the funding of safe AI and what role SIAI might play in the Singularity. He suspects that SIAI [...]</description>
		<content:encoded><![CDATA[<p>[...] AI researcher Shane Legg has some interesting thoughts on the funding of safe AI and what role SIAI might play in the Singularity. He suspects that SIAI [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Kevembuangga</title>
		<link>http://www.vetta.org/2009/08/funding-safe-agi/comment-page-1/#comment-19934</link>
		<dc:creator>Kevembuangga</dc:creator>
		<pubDate>Tue, 04 Aug 2009 04:57:29 +0000</pubDate>
		<guid isPermaLink="false">http://www.vetta.org/?p=558#comment-19934</guid>
		<description>Sigh...
Like all things &quot;Singularitarian&quot; the whole topic of &lt;i&gt;AI friendliness&lt;/i&gt; is utterly idiotic.
Friendliness cannot be ENFORCED upon any third party (of any kind, human, animal, robotic, ET aliens, whatever), friendliness only comes from shared values and peer recognition.
As  a demo please try to bring Talibans or jerks of the sort to  friendliness.</description>
		<content:encoded><![CDATA[<p>Sigh&#8230;<br />
Like all things &#8220;Singularitarian&#8221; the whole topic of <i>AI friendliness</i> is utterly idiotic.<br />
Friendliness cannot be ENFORCED upon any third party (of any kind, human, animal, robotic, ET aliens, whatever), friendliness only comes from shared values and peer recognition.<br />
As  a demo please try to bring Talibans or jerks of the sort to  friendliness.</p>
]]></content:encoded>
	</item>
</channel>
</rss>

