It's been an awesome journey since I started this blog nearly two years ago. My original goal was to understand exactly what we mean when we say that we want to "make the world a better place using technology". The answer is actually fairly complex - and I feel that very few people within the transhumanist community actually understand enough to have grasped this answer. This is because the answer is, in a way, disappointingly un-transhumanist. The non-existence of external, objective moral standards means that we are far less morally obliged to push a high-technology transhumanist agenda than we otherwise would be, and that even if the future does lead us to the high rates of technological change that transhumanists have so worshiped, the optimal way for us to use that technology will probably look quite mundane compared to the grand vsions of some transhumanists. As Michael Annisimov blogs:
"When we gain the ability to manipulate reality easily, most people will probably not choose to live within the sanitized white hallways of science fiction or the boring monoliths of The Jetsons. We will create more forests, rolling grasslands, huge gardens, splendid castles, and other things we can’t yet imagine. We’re all human, and most humans foster a romantic yearning to recreate some idealistic past. The true past was a place of disease and suffering, but we love the pleasant outlines transmitted to us through stories and our imaginations."
Mostly, the answer is contained in the work of Joshua Greene and Eliezer Yudkowsky's Values sequence on Overcoming Bias. It's funny that a mere year ago I was a staunch moral realist, and didn't appreciate the latter material. For example, buried halfway down the article on "the meaning of right" we find:
There isn't enough mystery left to justify reasonable doubt as to whether the causal origin of morality is something outside humanity. We have evolutionary psychology. We know where morality came from. We pretty much know how it works, in broad outline at least. We know there are no little XML value tags on electrons (and indeed, even if you found them, why should you pay attention to what is written there?)
If you hoped that morality would be universalizable - sorry, that one I really can't give back.
There are many pitfalls along the way to the truth about meta ethics and the "truth" about ethics. Not all of them are factual errors, either. It is entirely possible to completely dis-identify with human values, and identify entirely with technology. Since there are such things as convergent instrumental values, this is not obviously an incoherent position. Richard Hollerith maintains this position, and for a few very low months in my life I explored the implications of replacing naive morality with what one might call techno-worship. In the end, I realized that systems like GSZ trample over almost everything we find valuable, and I realized that this is hard to see because we take everything that is ubiquitous in our lives for granted. The existence of distinct persons. Love. Compassion. Humor. Sacred places and events (marriages, birthdays, celebrations...). Relationships.
For me, one of the most important is the tying of abstract scientific discoveries to specific places and people. Watson and Crick walking into the Eagle and announcing the discovery of DNA has so much more meaning than an anonymous computer of some kind making the same discovery.
To cut a long story short, the correct response to technology we see developing around us is to make technology serve our values, rather than making human beings serve technology. I might call this humanistic transhumanism. Unfortunately, this is a very hard message to sell... because humans like to "believe in" something that separates them from the crowd of other humans. Many humans who style themselves as "transhumanists" and "singularitarians" claim to hold ethical beliefs that are more there for show than for implementation. "I want to become a Jupiter Brain superintelligence", etc. [Write me an essay on the philosophical debates surrounding continuity of personal identity first... then see if you still want to]
Still, it is slightly easier to sell what I would call the rational utopia version of humanistic transhumanism; that world that is bounded below in goodness by Iain M Banks' Culture, that world that is approximated by Nick Bostrom's Letter From Utopia. That is something to dream about; a world that is as close to satisfying your muddled preferences as the rules of logic permit. I have wondered about how good such a world would be, but "pretty darn good, and optimal by construction" is a fair shot. The opportunities for intimacy, knowledge, personal development and growth, etc, etc would be pretty cool.
Slightly embarrassingly, it has taken me about 18 months to come around to the view of ethics that I was so skeptical about when Eliezer Yudkowsky first published the CEV concept on the SIAI Blog. Why is this? Am I stupid? I don't think so. I think that most really good hard scientists suffer from a fairly large degree of misanthropy; the classic image of the lonely physicist working on his own and never making contact with people is, in fact, true in many cases. In order to be the best possible mathematician/physicst, one spends most of one's time doing maths/physics. One loses contact with most normal human values, and I think that many such people become almost religious worshipers of science, or rather they dis-identify with many parts of their psyche that are not concerned with finding out scientific truths.
Goal System Zero or universal instrumental values is the logical axiology one would expect to be invented by a person who has dis-identified with every part of their mind that does anything other than do science.
So, I would diagnose myself has having made an error* of the heart - of being bullied too much at school, of finding social and interpersonal skills so late in life, and of being put under such pressure by the way I was brought up and by a childhood environment based upon conditional (rather than unconditional) love. Luckily for me, I also have a sex drive, and once you have a sex drive you have to go out and relate to people. And once you go relate to people, you realize that there really is more to life than math and science! It is interesting to note that my rejection of the techno-worship, GSZ/UIV stance roughly coincided with the first time in my life that I realized that someone had loved me unconditionally. She knows who she is. There are special moments between you and another human being, using Haidt's terminology there are moments of secular sanctity, moments where the mere atoms that make up the world fall into a configuration that we attach special significance to. And yes, most, if not all of these moments are a just instantiations of games that evolution programmed us to play. But, to quote Stathis from the SL4 list:
Yes, you could say that it is all just a game evolution has programmed us to play, but it's an important game for those so programmed.
This is such a subtle and beautiful message, and many people could learn a lot from it.
* (Strictly speaking this is not an error. As hume put it: Tis not contrary to reason to prefer the destruction of the whole world to the scratching of my finger. )
Posts
34 comments:
I'm happy you got so much out of this. Did you find the romance you mention recently?
With respect to learning about meta-ethics and ethics, this might be a good time to caution against some of the ways we can go astray in the enthusiasm of a good new idea. This is an especially important danger if you have been introduced to a field through a single source. Greene's dissertation may have gotten first crack at introducing much of academic meta-ethics and moral psychology to you in one fell swoop, and Greene's own additions to that corpus may get excess credibility points if you don't adjust for it. Likewise, Eliezer has written excellent introductions to a number of important research literatures, but someone who has only been exposed to those literatures through OB may wind up with less of an understanding of how someone could viscerally hold the alternative views.
Reading some moral realists like Railton in their own words (or listening to them in this bloggingheads on moral realism
http://bloggingheads.tv/diavlogs/13443) might help to understand better why Greene's views haven't quickly converted the whole of the philosophical academy. I would also recommend Chalmers' "The Conscious Mind."
It's also important to remember the extent to which these fields still have open problems with significant importance. For instance, it's very uncertain whether there is such a thing as a 'Collective Extrapolated Volition' for humanity that doesn't depend very sensitively on the parameters of the extrapolation and aggregation. Issues like the Arrow Impossibility Theorem from voting theory, the malleability of focus for evolved moral sentiments, and the interaction between our number sense and motivation create looming challenges for such an approach.
Actually, scratch the Bloggingheads unless you're doing laundry and need something to listen to, papers are better.
I'm still sceptical about this view.
I don't feel that I understand morality and associated problems well enough to rule out the existence of a deeper more universal kind of morality. For example, I can't say for sure what pain is, except in terms of my own subjective experience. I feel far to ignorant to be able to really adhere to any specific position on morality.
Secondly, I don't think your position will be sustainable. Unless you really limit people's freedom and lock down their cognitive internals, people in the future are going to have quite different values than people today... assuming that they can still really be called "people" a hundred or thousand years into the future.
Thanks for your comment, Carl.
As for the romance, that was an odd affair that took a while to sink in, but started a long while ago.
"might be a good time to caution against some of the ways we can go astray in the enthusiasm of a good new idea"
- yes, indeed. Good intellectual work is inherently a cautious process, but for now I am prepared to forgo the requisite caution in favor of the emotional boost that wholeheartedly holding this position gives. [yes, this is potentially interpretable as self-delusion]. In good time I'm sure I'll start asking the obvious questions that you bring up: why are there still philosophers who hold realist positions (probable answer: the word realist can mean different things to different people, and in philosophy those meanings can become very complex).
Also, it is emphatically the case that formalizing human values is going to be an *incredibly* tough task.
"For instance, it's very uncertain whether there is such a thing as a 'Collective Extrapolated Volition' for humanity that doesn't depend very sensitively on the parameters of the extrapolation and aggregation"
- indeed. In fact, given the nature of human choice ("the choices you make affect you, and you affect the choices that you make") that any extrapolation of a human personality or group is probably going to display divergent behavior. But these are problems one can only even see once one has made the realizations detailed here.
Shane: "I'm still sceptical about this view.
I don't feel that I understand morality and associated problems well enough to rule out the existence of a deeper more universal kind of morality."
- We should chat about this some time. Briefly, the challenge is in pinning down what the naive sense of morality we have is supposed to refer to, and realizing that our realist attitude is something of a cultural construct. Joshua Greene does this best in his dissertation.
Once you have the evo-psych explanation for naive morality, you realize that any formal concept that corresponds to the naive concept *has* to be antirealist. The UIV concept that I came up with for a "canonical axiology" is an example of a well-defined (well..) formal notion of axiology, but fails to match up closely to our naive notions. Kant's Categorical Imperative is another attempt that fails in different ways, though to go into that debate in a blog comment would be truly hilarious...
Good post. I've had similar thoughts for a long time about why people hold bizarre transhumanist ethical views.
I second Carl's warning against enthusiasm and single sources (I think I have considerable problems with this myself).
Roko,
There is no reason for thinking that these latest views of yours are any more accurate than the earlier ones I'm afraid.
This in particular caused pearls of laughter to escape my lips:
"Mostly, the answer is contained in the work of Joshua Greene and Eliezer Yudkowsky's Values sequence on Overcoming Bias."
Um, No. Giveen that Yudkowsky is a Libertarian (a crackpot political philosophy which says that all public human interactions shoud ideally be reduced to market exchanges), few would regard him as a reliable source of moral wisdom.
I also chuckled at the Stathis quote. I recall debating him on the everything-list, where he argued that 'the laws of physics' don't exist (he was also an anti-realist about physics).
---
All I can do is point to the outline of the 'ontology-scape'.
"You should see what I see."—Johnny Smith (Dead Zone).
Of the three top-level ontological domains (Physical, Volitional, Mathematics), there's strong evidence for a realist position in two of them.
We see in the physical domain, that there apparently do exist universal 'laws of physics' , which cannot themselves be located in space and time (they are abstractions).
We also see in the mathematical domain, that there apparently do exist universal 'mathematical forms', which cannot themselves be located in space and time (also abstract).
Reasoning by analogy, there should exist 'universals' in the domains of values also.
Why should the volitional domain (concerned with values) suddeny be the exception? Taking a non-realist position when it comes to values introduces a severe 'symmetry-break' in the ontology-scape.
Of course, the non-realist can restore symmetry to ontology by going the whole way and denying that physics and math is universal either (aka Stathis). But this is even less credible.
I apply Occam's razor here. The ontology-scape should be simple and smooth, it should not have a big jagged piece where the box 'Universal Values' ( next to the boxes 'Universal Laws of Physics' and 'Universal Pure Math Forms') has been cut out.
Eliezer used to be more libertarian (http://www.overcomingbias.com/2007/03/policy_debates_.html?cid=62131998#comment-62131998), although I don't know that he would ever have self-identified as Libertarian, but I *really* don't think that he would ever have endorsed the reduction of all public human actions to market exchanges as ideal. Marc probably knows this after years at SL4 and Overcoming Bias.
Marc: "I apply Occam's razor here. The ontology-scape should be simple and smooth, it should not have a big jagged piece where the box 'Universal Values' ( next to the boxes 'Universal Laws of Physics' and 'Universal Pure Math Forms') has been cut out."
Well it is certainly a simple hypothesis. But don't forget that occam's razor must be used in conjunction with experimental verification of hypotheses. And the data is in: "ethics" is what it is because of chance events in the history of human evolution, and then chance events in the development of western culture.
As I tried for to do for a while: you can find "objective" axiologies, but the constraint of them being objective means that they cease to have anything at all to do with our naive concepts of good/bad outcomes. See the posts on UIVs and critiques thereof.
In which case you're not really talking about ethics. Basically, this post could be titled "game over for objective ethics", though Hume realized this a rather long time before me...
Roko:
Yes, we should chat sometime.
Basically what I'm arguing is: if I really truly understood things like love and pain then I wouldn't be surprised if that understanding had a significant impact on my ethical views. There may be a deeper logic to ethics that I am currently aware of in my rather ignorant state.
Roko,
There's no such thing as 'chance events' in a lawful universe.
And there are additional data points that must be looked at here.
Shane may be on the right track here. 'Valuation Qualia' (conscious experiences with impicit value judgements, such as pleasure/pain) have a peculiar character, which has not yet been explained.
Whereas physical modalies (for instance vision) seem to have a potentially infinite variety, valuation qualia are scalar, binary and distinct. Further, analogues to human emotions occur across the animal kingdom. This is suspicious and suggests some 'universal' principle at work.
Marc: "Further, analogues to human emotions occur across the animal kingdom. This is suspicious and suggests some 'universal' principle at work."
- yes, it's called evolution.
"'Valuation Qualia' (conscious experiences with impicit value judgements, such as pleasure/pain) have a peculiar character, which has not yet been explained."
- conscious experience is, I suppose, they last and only relevant mystery. But it can't (as far as I can see) change the arbitrariness of our current values. It can, perhaps suggest what the likely evolution of those values are.
So are you and shane to be known as the "Realist Crew", eh?
Roko: I would prefer to call you a "faith based anti-realist".
I'm much too ignorant and lacking in faith to commit to either.
Shane: "Roko: I would prefer to call you a "faith based anti-realist"."
- I should set out the argument I have seen in some detail. This will be a future blog post topic - thanks
R
>- yes, it's called evolution.
You seem to have missed the point about the sharp distinction between valuation qualia (which seem to have distinct common charcteristics) and other types of conscious experiences (which really do seem to be arbitrary).
>So are you and shane to be known as the "Realist Crew", eh?
I just feel obligated to warn you about Eliezer Yudkowsky, which seems to be a source you're relying too heavily on.
In my opinion, he doesn't understand a thing about human nature, and is dead wrong in virtually all of his 'big ideas' (except perhaps for the 'hard take-off Singularity' idea itself).
If I may summarize my current views:
(1) Bayesian Induction is *not* the foundation of rationality. Analogy formation is.
(2) Liberty/Volition is *not* the foundation of morality. Aesthetics is.
(3) True general intelligence without consciousness is impossible.
(4) I advocated 'universal values' *not* 'objective values'. I *do* agree that an AI would *not* be ethical automatically, and intelligence alone *cannot* justify ethics.
(5) The UAI threat does *not* follow from (4) at all, contrary to what SIAI is claiming. Their mistake is to assume the independence of the goal system and the intellect. I'm saying the AI's ability to self improve *depends* on the goal system.
And my proof? It's all in my Ruby code ;)
Roko: Cool. I look forward to it!
Remember:
Ruby Rules ;)
http://www.wisegeek.com/contest/what-is-ruby.htm
Of the three top-level ontological domains (Physical, Volitional, Mathematics), there's strong evidence for a realist position in two of them.
I disagree. I think one of them, physical, has evidence for a strong realist position, though not in the same way you lay out.
We see in the physical domain, that there apparently do exist universal 'laws of physics' , which cannot themselves be located in space and time (they are abstractions).
What we refer to as the 'laws of physics' are an intellectual construction created by human minds. That the physical world is real is a different issue. One can make a solipsist argument against the physical world, which I think is rather weak, but arguing for the existence of a set of abstract 'laws of physics' being a real thing is a different matter all together.
We also see in the mathematical domain, that there apparently do exist universal 'mathematical forms', which cannot themselves be located in space and time (also abstract).
This is where I bring up a more fundamental disagreement. Mathematics as a discipline is a result of human consciousness. To assume from a sample size of one (human consciousness) that all possible forms of consciousness would have conforming "mathematical forms" is unwarranted extrapolation. To use language as an analogy, it is entirely possible that dolphins have a language that doesn't conform to human language rules. It is also possible that there is some race of extra terrestrial beings that have a form of consciousness that is fundamentally different from humans could develop a fundamentally different form of mathematics.
Reasoning by analogy, there should exist 'universals' in the domains of values also.
My short response is a simple "no". My long response is that the domain of value is completely different than the other two domains you cite. With the physical world we have direct access as individuals to the physical world. The best explanation as to why our experience of that world are in sync is that there is in fact a physical world that acts in a certain way. In the case of the mathematical world, there are a number of possible explanations. First, the strong realist position: There are mathematical truths in the world. The second being that being that our understanding of mathematics is based on our shared properties of human consciousness. This is the view that I hold. The third is that mathematical truths are a social construct. I think the last is easily dismissible but that the first two are equally likely given the evidence we have.
The domain of values however is a different. We have widely disparate examples of systems of valuation. In the domain of aesthetics this is most obvious to most people. I don't think anyone here will argue that there is an objectively "most handsome male". The handsomeness of any given male is based on societal standards. Unlike in the physical domain we have no access to the actual evidence of the existence of aesthetic phenomena. Likewise we have no access to the actual evidence of the existence of mathematical phenomena, but we do have access to a vast oeuvre of mathematical writings that are logically consistent, and that people will generally admit are wrong when they are not logically consistent. In the domain of values we have no such thing. We have some work done on the human proclivity to value symmetry but we lack a non-binocular vision view on aesthetics that has been explored in depth. We also lack, as in the mathematical domain, access to the "truths" of valuation.
Hi, Coathangrrr
Good well thought-out reply! Wrong of course, but well thought none the less ;)
---
Before discussing your points I just want to quickly go back to the points of Roko and Anissimov again, since they are both clearly confused.
Roko, all the philosophy of Hume says is that ethics doesn't follow from intelligence. I AGREE with this. Intelligence doesn't result in ethical behavior.
But a non-realist position on ethics is a *stronger* claim than this. That is to say, a non-realist ethical position *doesn't* follow from Hume (although admittedly Hume does make a non-realist position more likely).
The example of Phineas Gage actually supports my position. When Gage lost the part of his brain responsible for ethical behavior, he ALSO lost the cacapity for reflective decision making (Note: He became 'shiftless' and unable to follow long-term goals).
Studies of psychopaths provide further confirmation. Although its true that lack of ethics does not affect ordinary intelligence, it DOES affect the capacity for reflective decision making: again, pyschopaths have documentated reflective deficits, namely they can't distinguish negative emotions.
All this shows that the goal system is NOT fully independent of intelligence in the manner Anissimov is claiming.
---
Back to Coathangrrr. I think the crux of our argument is where you try to deny that 'the laws of physics' are objectively real. I think you are badly mistaken there and if you talked to most physicists they would strongly dispute your claim.
There's a very sharp distinction in physics between the *laws* themselves and *the boundary conditions* (the arrangements of physical matter).
Of course human *knowledge* of these laws may be incomplete or inaccurate. But I think the 'laws of physics' are a real objective abstraction that exists 'out there'. And once this point is accepted, there's a much closer analogy between physics, values and math than you seem to think.
We have no direct access to the 'laws of physics' either, so the physical world isn't really so different from the world of values or math.
--
In the case of values, I think you are failing to distinguish between different levels of abstraction (as with the distinction between 'laws of physics' and 'physical matter).
--
For instance, in aesthetics, the actual *artifacts* (concrete works of art) are human creations, no one is disputing that. But this is precisely analogous to *physical* artifacts in physics (human engineering feats for example, which not one argues are 'God given' but are human inventions).
At a higher of abstraction, the *process* of making or thinking about art depends on the human mind (again no argument there). But again, this is precisely analogous to applied rules in physics (human engineering heuristics).
Jump to a still higher of abstraction and there is a perfect analogy between 'laws of physics' (objectively real) and 'aesthetic styles or forms' (which I argue, are also objectively real).
Studies of psychopaths provide further confirmation. Although its true that lack of ethics does not affect ordinary intelligence, it DOES affect the capacity for reflective decision making: again, pyschopaths have documentated reflective deficits, namely they can't distinguish negative emotions.
What you are talking about is a correlation between reflective thinking and ethical action, not anything having to do with the existence of real ethical truths. Reflective thinking is necessarily required to act upon a system of ethics one holds because the system is abstract and situations are specific, thus one must compare the abstract and the specific and decide what to do. That's impossible without reflective thinking.
I'm not sure how that relates to your argument on whether valuation is, or can be, objective or not.
Back to Coathangrrr. I think the crux of our argument is where you try to deny that 'the laws of physics' are objectively real. I think you are badly mistaken there and if you talked to most physicists they would strongly dispute your claim.
I think they would, but for the wrong reasons, mostly because they would think that I meant that the laws of physics are false, which is not what I mean. I mean that the laws of physics have no real existence, they are a description of the behaviour of the physical world and while they may be true and give an accurate representation of the physical world, one that allows us to make predictions of the world, they are not real.
We have no direct access to the 'laws of physics' either, so the physical world isn't really so different from the world of values or math.
We don't have direct access to the laws of physics, but we have access to the things those laws describe, matter, energy, etc. What we don't have access to, in math or ethics, is any access to the objects described by the systems.
Ethics describes the goodness or badness of acts. We don't have access to the goodness or badness, or rightness or wrongness, itself, just our beliefs and feelings about what is right or wrong. But we obviously don't have the same fundamental disagreements about physics as we do about ethics. No one, or almost no one, is going to argue that a dropped pen will fall to the ground on earth. In contrast, people have different conceptions of what acts are ethical and which aren't as well as differing on what is good art and what isn't. Some of those disagreements are a function of disagreements on matters of fact, a lot I'd wager, but some are much more deep set. Ultimately we can never have access to the rightness or wrongness of an act, only our belief or feeling that an act is right or wrong. That is the difference.
For instance, in aesthetics, the actual *artifacts* (concrete works of art) are human creations, no one is disputing that. But this is precisely analogous to *physical* artifacts in physics (human engineering feats for example, which not one argues are 'God given' but are human inventions).
Again, ethics and aesthetics isn't working with the artifacts, it can't because the artifacts can't contain beauty themselves, beauty is something perceived by the viewer, not in the artifact. In physics, the actions of particles and such are what is studied, not our differing feelings about those actions.
Jump to a still higher of abstraction and there is a perfect analogy between 'laws of physics' (objectively real) and 'aesthetic styles or forms' (which I argue, are also objectively real).
Another problem with your argument here is that the existence of an objectively real Aesthetic would mean that someone could be wrong about something being beautiful. I don't think anyone could argue that in a coherent manner. If I find something beautiful it is beautiful to me, unlike the case in physics where I may think that some set of laws is the case and in fact that set of laws in not the case. In physics one can make predictions about how particles will act, and those predictions will be true or false. In aesthetics one might be able to make predictions about what people will find beautiful, but those predictions will not be proved or disproved because people's views on the beauty of any specific artifact will vary from person to person. If there were an objectively real aesthetic standard then this would not be the case.
>I'm not sure how that relates to your argument on whether valuation is, or can be, objective or not.
Without reliable reflective reasoning, no mind can continue to improve itself, and it would be neccesserily be limited.
If, as I claim, the goals and the reflective abilities of a mind are related, then, in the limit that minds improved themself to the super-human level, the goals of all minds would have to converge. This would prove universal values.
Remember, the data suggests that lack of ethics disrupts reflective decision making. This means that no AI could maintain a stable goal system under reflecion unless that AI was already fully ethical. This provides evidence for universal ethics.
>I mean that the laws of physics have no real existence
The laws of physics continue to operate even in the absence of any concrete physical objects. They have to explain the origin of the physical unverse itself. That's why physicists think that the laws of physics have to be objectively real and not just a human construct.
According to big bang/inflation theory, close to the very beginning of time there was no matter, only energy and law ( pure abstractions). This means that these abstractions have to be every bit as real as concrete objects.
>We don't have direct access to the laws of physics, but we have access to the things those laws describe, matter, energy, etc. What we don't have access to, in math or ethics, is any access to the objects described by the systems.
I disagree. The 'objects' of math are computer programs/software - that's exactly what software is, concrete math - so I claim that we DO have direct access to 'math' objects!
Similiarly I say that the 'objects' of ethics are simply all the things that can be interpreted icons (signs, symbols etc etc, anything representing intensionality). Again, I claim that we DO have direct acess to 'ethical' objects!
As I told you, I don't claim universality on the object level, only at a higher level of abstraction.
>Again, ethics and aesthetics isn't working with the artifacts, it can't because the artifacts can't contain beauty themselves, beauty is something perceived by the viewer, not in the artifact. In physics, the actions of particles and such are what is studied, not our differing feelings about those actions.
Again, I don't think you are properly distinguishing between levels of abtraction.
There is a level in physics in which we have to deal with human perceptions also...the secondary properties such as color etc. At a low level of abstraction, physics is just as dependent on human perception as aesthetics is (think virtual reality!).
Your claim that 'the artifacts can't contain beauty' is simply stating your own assumption that aesthetics is not objective.
>In aesthetics one might be able to make predictions about what people will find beautiful, but those predictions will not be proved or disproved because people's views on the beauty of any specific artifact will vary from person to person. If there were an objectively real aesthetic standard then this would not be the case
Yes this is the best argument against objective aesthetics, there seems to much less agreement than in physics.
But here, I think you are pushing the analogy between physics and values too far... only physics is about prediction, the domain of values is not about prediction.
The platonic values can't be in the form of 'laws' (unlike physics) since obviously we can choose not to be ethical.
The evidence for universal values is my arguments that goal systems converge as minds self-improve (see my discussion at beginning of post). So I see the universal values as 'ideals' (a sort of 'ethical limit') rather than laws.
Without reliable reflective reasoning, no mind can continue to improve itself, and it would be neccesserily be limited.
If, as I claim, the goals and the reflective abilities of a mind are related, then, in the limit that minds improved themself to the super-human level, the goals of all minds would have to converge. This would prove universal values.
So you are making an empirical claim. I, obviously, think you're wrong about that. I also think you fall into the same trap as Aristotle and Mill, assuming that mental activity, reflective thought, is the greatest type of activity. I'd also disagree with this. I think it is for a great number of people, though what percentage I wouldn't dare to estimate, but to assume that thinking more is better is completely unfounded. Yes, people who have better reflective reasoning generally behave in a way more consistent with what we consider ethical, but our beliefs about what is ethical was created and developed by deep reflective thought. It is basically begging the question. Reflective thought is good, therefore it can tell us what is good. Perhaps this is a misunderstanding of your argument, but I've seen similar quite often.
Remember, the data suggests that lack of ethics disrupts reflective decision making.
No, it suggests a correlation between a lack of ethics and reflective decision making. It is entirely possible that it runs either way, that a lack of reflective thinking affects ones ethics. In fact, it seems much more likely to run in this direction.
This means that no AI could maintain a stable goal system under reflecion unless that AI was already fully ethical. This provides evidence for universal ethics.
I assume you mean that it *would* provide evidence for universal ethics, not that it does, because our distinct lack of ethical AIs means that you are making an as yet untested empirical claim.
The laws of physics continue to operate even in the absence of any concrete physical objects. They have to explain the origin of the physical unverse itself. That's why physicists think that the laws of physics have to be objectively real and not just a human construct.
There are two problems here. First and foremost is the assumption that there were abstract physical laws at work during the formation of the universe. Physics takes the physical laws which we abstract from observation today and uses those as a basis for determining what the universe was like in its initial stages. They have no proof that those laws, or some set of laws derived therefrom, were in effect at the time. Or really whether any laws were in effect at the time. They assume that there were laws some how related to the ones we have now at the time because without that assumption they would have nothing to say about that time of the universe. The point is that we don't and can't know about what laws there were then or whether there were laws at all, we can simply assume that the laws of physics are universal and work from there.
I disagree. The 'objects' of math are computer programs/software - that's exactly what software is, concrete math - so I claim that we DO have direct access to 'math' objects!
The objects of math are numbers, abstract entities to which we lack access. The manipulation of symbols, which is what software is, is not an object, it is a abstract process that returns a set of symbols to the user that will hopefully have meaning or utility to said user. Software does not manipulate numbers, it manipulates symbols. If you think of math objects as wholly symbolic then that undermines your whole analogy between physics and math. Symbolic objects are not, by definition, real. They are representations of some real thing or they claim to be a representation of some real thing.
Similiarly I say that the 'objects' of ethics are simply all the things that can be interpreted icons (signs, symbols etc etc, anything representing intensionality). Again, I claim that we DO have direct acess to 'ethical' objects!
The problem here is that there is no good definition of what falls into the set of "ethical objects". We have no way to determine that set because we require an ethic to do so, thus begging the question. For example, there may be something that you consider to be ethically symbolic, and I'll admit that I'm a bit iffy on what an ethical sign or symbol would be, and yet there are plenty of others who don't consider it to be ethically symbolic. What you seem to be claiming is that there is a universally recognizable, to the suitably reflective thinker, set of ethical objects that are good or right and another mutually exclusive, set of ethical objects that are bad or wrong. If the full set of ethical objects is indeterminable, as I claim, then we could never know the whole of either of those sets and thus there could be no universal ethic.
As I told you, I don't claim universality on the object level, only at a higher level of abstraction.
I think this is a fundamental difference in our views. In my ontology the higher level of abstraction the less real something is. You are claiming that given a high enough level of abstraction we can find a universality of ethics(and math and physics). But, if that is true, and this is why reflective thinking is so important to your argument, then what you are looking for is a universal explanation of actions and beliefs, and assuming it is there because you claim it is there in physics and math. But it isn't there in math, see Goedel's incompleteness theorem if you doubt that, and it isn't there in physics because the abstract laws of physics rely on math.
Again, I don't think you are properly distinguishing between levels of abtraction.
There is a level in physics in which we have to deal with human perceptions also...the secondary properties such as color etc. At a low level of abstraction, physics is just as dependent on human perception as aesthetics is (think virtual reality!).
Perhaps I'm not distinguishing enough between levels of abstraction, but what I hear you saying is that the study of math and ethics is based on symbols, which are abstract, whereas the study of physics is based on physical objects, which are not abstract. Let's not get into color. I'm definitely an anti-realist in regards to color.
I am confused by your mention of artificial reality. Our current incarnation of virtual reality is an object illustration of the laws of physics. It uses the laws of physics to confuse our perception. In fact, it is an object illustration in how different physics is from the other two domains. The idea of a mathematical or ethical virtual reality is rather absurd. How could you make an artificial reality where you seem to add two and two and yet really do not. Or, how could you you make an artificial reality where some action which isn't ethical, assuming realist ethics, is ethical?
Your claim that 'the artifacts can't contain beauty' is simply stating your own assumption that aesthetics is not objective.
Good point. I was under the assumption that no one accepted that sort of realist view of art, consider me corrected.
Yes this is the best argument against objective aesthetics, there seems to much less agreement than in physics.
But here, I think you are pushing the analogy between physics and values too far... only physics is about prediction, the domain of values is not about prediction.
No, I am pushing *your* analogy to the point where it breaks down. My point is that there is not an analogy between the three domains you claim to be analogous. If there are universal truths about ethics, or any system of valuation, that are not predictive then those universal truths are useless. What good is it to know universal truths about ethics if I cannot use those to predict whether a specific action I might take would be ethical or not? They wouldn't be. That means that an objective ethical framework *must* be predictive.
The same holds with any system of valuation. If I know some sort of universal laws of beauty and yet can't use those to tell me what is beautiful then It would be useless. This is also true in the realm of politics and any other realm that includes valuation.
The evidence for universal values is my arguments that goal systems converge as minds self-improve (see my discussion at beginning of post). So I see the universal values as 'ideals' (a sort of 'ethical limit') rather than laws.
Again, you are begging the question. to say that minds improve requires a preexisting method of valuation for what improvement is, because improvement implies being better, which is a valuation.
That got pretty long there, but this is a really good conversation.
>That got pretty long there, but this is a really good conversation.
I just want to clarify a couple of points.
>what I hear you saying is that the study of math and ethics is based on symbols, which are abstract, whereas the study of physics is based on physical objects, which are not abstract.
No. I'm saying that physics, math and values can be devided into different levels of abstraction, and all three domains have both abstract and concrete elements. (see below).
>The objects of math are numbers, abstract entities to which we lack access.
Numbers are the *high level* (abstract) objects of math. (analogous to 'laws of physics')
I'm saying that there are also concrete *low level* math objects, (analogous to physical objects), and I''m suggesting that software is it.
>The manipulation of symbols, which is what software is, is not an object
A process can be defined as an object. Running software has an associated *design* (the static computer program), the representation of that program *is* an object.
>If you think of math objects as wholly symbolic then that undermines your whole analogy between physics and math.
No. See above. Only the *low level* math objects are symbolic human inventions. (the software). The *high level* objects (numbers) are the ones that I'm saying are the objective universals.
>I am confused by your mention of artificial reality.
Virtual reality is a *representation* of the physical world as it looks to us humans. But this wholly a construction of the human brain.
My point was that our *perceptions* of the physical world depend on the human brain, just as certain aspects of aesthetics do, so there's not such a great difference between the domain of physics and values.
>The problem here is that there is no good definition of what falls into the set of "ethical objects".
True. Thats why I tried to reason by analogy with physics and math.
Also remember that the domain of ethics is only a sub-set of the domain of values, so I should really have said 'value objects'.
I think the *low-level* 'value objects are all our human signs and symbols the alphabet for instance), since they represent 'intensionality'. (The field that studies signs and symbols is called semiotics by the way). Of course these are not objective universals. But higher level 'value objects' might be.
Regards
Marc
Also remember that the domain of ethics is only a sub-set of the domain of values, so I should really have said 'value objects'.
Very true, while writing I kept thinking I should just talk about the domain of values rather than switch between values and ethics.
No. See above. Only the *low level* math objects are symbolic human inventions. (the software). The *high level* objects (numbers) are the ones that I'm saying are the objective universals.
This is where I have a problem with your account. I don't think that a system such as math, where the bases for proofs are symbolic and the "real" things are abstract, which is also true of your account of the domain of values, is comparable to the domain of physics where the objects on which we base proofs are not of that same kind. Talking about the abstracts in physics, the "laws of physics", makes it even more problematic because those abstracts are based on another domain, that of math.
Ultimately, I don't see that you can establish a coherent analogy other than of the broadest sort between the three domains. And you need to be able to have a more specific analogy to be able to argue this point from analogy.
I don't think this is going to be sorted out here in the comments, but if you have suggestions for further readings on the subject I'd like to look into it further.
I have flirted with a moral-realistic argument for my position but it is a non-essential aspect of my position, and I decided to stop flirting with it, so like Roko and Eliezer I am not a moral realist (though like Roko and Eliezer I was at one time) and Roko concurred in private email that I am not a moral realist.
I admit that mine is a bloodless, detached, abstract way of looking at morality, and that is probably a big part of why it has so few adherents among the humans.
Roko writes:
I would diagnose myself has having made an error of the heart . . . Luckily for me, I also have a sex drive, and once you have a sex drive you have to go out and relate to people. And once you go relate to people, you realize that there really is more to life than math and science! It is interesting to note that my rejection of the techno-worship, GSZ/UIV stance roughly coincided with the first time in my life that I realized that someone had loved me unconditionally. She knows who she is.
First, let me say that I do not worship technology or science.
It would a huge mistake for most men not to have the love of a good woman, and the easiest way to arrange for that is to have sex with a good woman.
I had the love of a good woman when the moral system I am now calling goal sytem zero became important to me. I have had the love of another good woman since then, and that experience did not cause me to change my moral position.
Then there is this John David Garcia fellow who introduced me to what I now call goal system zero. Since John is dead whereas I plan on dating more, I will be freer with information about John's sexual history than my own.
Very sexually active in college. By the time he graduated he was already married to the woman who would become the mother of his children. At the age of 35 or so, decided to devote the rest of his life to spreading goal system zero. After his children went off to college, he decide to live in Chile and then in Mexico because Chileans and Mexicans seemed more receptive to goal system zero than Americans seemed. Since his wife chose not to follow him and stayed in San Francisco, he took mistresses in Chile and Mexico (without deceiving anyone). I got the impression from my talks with him about sex that John was sexually active almost the whole time from puberty to when he died age of 64 or 65, still fully committed to goal system zero.
I mention this so that someone reading Roko's experience will not leave with the impression that committing to goal system zero is incompatible with a satisfying love life.
Richard Hollerith: "I mention this so that someone reading Roko's experience will not leave with the impression that committing to goal system zero is incompatible with a satisfying love life."
- I think it is. We obviously differ somewhere.
Firstly, let me just get this clear: GSZ entails the belief: "I place no terminal value on human beings, including my wife/partner". Do you agree with this?
If you do, then you seem to be committed to the following position: if you found a powerful AI lying on street waiting for a goal system to be plugged in, you would be compelled (by your loyalty to GSZ) to put GSZ in as the goal.
This would directly cause the death of your so-called "loved" one, and you, and anyone else who is dear to you.
To me, this seems like non-love; though perhaps I am falling into the trap of arguing semantics.
I would, at the very least, program an FAI to dedicate 0.1% of the mass and energy of our future light cone to the implementation of a human friendly "sysop", leaving 99.9% for a GSZ AI to build ever more complex structures and optimizers. If you don't support this compromise, then we are too far apart to come to any agreement, your intuitions are just too bizzare.
Coathangrrr,
We'll leave it there I think; I think you put your finger on it when you said:
I think this is a fundamental difference in our views. In my ontology the higher level of abstraction the less real something is.
Yes, I believe the exact opposite. I think that the higher the level of abstraction, the more real something is.
I don't see that the physical world as it appears to us is any less symbolic than values or math. As I mentioned, I think the physical objects we see around us are really only constructs of the human brain whose job it is to conjure a 'virtual reality' as a model, and only 'the laws of physics' are real. This makes me an out-and-out platonist.
Richard said:
It would a huge mistake for most men not to have the love of a good woman, and the easiest way to arrange for that is to have sex with a good woman.
Romantic matters have no relevance to logical claims.
Then there is this John David Garcia fellow who introduced me to what I now call goal system zero
Zero is an ironically apt description it. Why so people on the net appear to fly off on such bizarre tangents is an interesting sociological topic.
--
All my ideas are implemented in code. And that, I think, is what all net intellecuals should be doing more of, coding, rather than pontificating. Coding forces one to enact a precise logical description of ones ideas you see.
--
Those in posessesion of the secrets of the universe don't need podcasting, twittering and blogging. Do you really think that a transhuman intelligence is going to rush off to 'Less Wrong' trying to earn 'Karma points'? Think about it. Why are Yudkowsky and co doing this? It's precisely because their real position is so weak.
A real transhuman would just hack the 'Less Wrong', make it itself the sole contributor, assign itself infinite Karma points, and ban all the idiots, which would be nothing than they deserve. Then a new 'blog series' would be written, but the presentations would be in immersive virtual reality.
My final comment on AGI: Keep an open mind, question eveything, don't believe anything you read on the net unless its been enacted and tested in code first.
Good Luck
-Marc
Let me address your final paragraph first. Having anticipated your concern, I wrote the following 2 days ago. Sounds like you have not seen it yet.
Also note that I am very open to compromise (and will ally myself only with singularitarians who I judge to share my openness to compromise). Although GSZ does not particularly want to keep the humans alive, it does not particularly need to kill or to frustrate them except where a human plan or a human ambition or a human desire would hog most the resources needed by GSZ. Consequently it should be relatively easy for an alliance of the humanists and the adherents of GSZ to steer reality into a future that is very satisfying to both.
To anyone following this dialog between Roko and me: part of it is in a comment section at my blog.
I understand very well that the vast majority of the intellectual and scientific firepower directed at the singularity is in the hands of humanists, and consequently it is I and not the humanists who will be doing most of the compromising if there is a compromise. In particular it is I who will need to brainstorm on how my moral concerns might fit into their plans, which as you know revolve mostly around CEV, and not the other way around.
Roko, the attention span of a reader on the web is quite brief. Since I have defined my role to be that of an advocate for a certain moral position, of course I want as many people as possible in my target audience to consider my messages. Consequently I have taken every opportunity to minimize the number of words I use and to keep my images concrete and readily accessible rather than abstract.
One image that is concrete and readily accessible to any human reader is that of an adversarial relationship between two people. There is only one future light cone, and two adversaries have very different ideas of what should happen to it. Nice, concrete image! I have mostly neglected to mention that reality is probably structured in such a way that both Eliezer and I can get most but not all of what we want for the world. And I have mostly neglected to mention that I am very willing to compromise. Mentioning those two things would have added extra words to my messages and muddied up one of the most useful and accessible mental images I have available to me. But of course I am willing to compromise!
Roko, you write: if you found a powerful AI lying on street waiting for a goal system to be plugged in, you would be compelled (by your loyalty to GSZ) to put GSZ in as the goal.
This would directly cause the death of your so-called "loved" one.
Yes, but.
Yes, my loyalty to GSZ and only GSZ leads to the death of my "loved" one in some situations.
But it is also true that if I cared for the survival of my loved one and nothing else then in some situations I will be led to do things that even you would disapprove of. Is it not true for example that it increases the probability of the survival of my loved one for me to create billions of copies of her and scatter them as widely as possible? And of course each copy should be surrounded by an armada of starships dedicated to her protection.
The more precise or unambiguous a proposal for the goal system of the powerful AI, the easier it is to identify a situation in which that goal system will violate your moral intuitions. It just so happens that right now out of all the proposals I know of, mine is the most precise. (But not precise enough! My next priority is to define a more precise goal or end that I definitely care about more than the survival or the flourishing or the happiness of the humans.)
I think precision is very valuable in this role. I wish the humanists would advance a more precise proposal than the very imprecise CEV so that I can start looking for an elegant compromise between their proposal and mine. I wrote a little more about this 2 days ago over on my blog. Let me add that the worst imprecision in CEV arises from the E part: "if we thought faster, knew more, were more the person we wished we were". How that part is made more precise has drastic effects on how compatible CEV is with something like GSZ. I should say more about this, but what I want to say is too many words to fit comfortably in a blog comment section.
Do you still think that GSZ is incompatible with a satisfying love life?
If so, do you think something like the following?
Most women who are having sex with you or who are contemplating having sex with you want to know everything about you. Moreover, to obtain the important and massive benefits of having the love of a good woman, you have to keep her updated her on everything important that has happened to you, what you have been thinking about and what your hopes and plans are.
Do you maintain that someone like me who spends a significant amount of his time advocating for GSZ cannot share that part of his life with his loved one without eventually frightening her, alienating her or causing her to force him to choose between GSZ and her?
Well, my experience has been that that it is not a problem in practice: I have been able to talk openly about my participation in the public discourse over the singularity with the woman I have been seeing without harming the relationship.
There are many many things that can harm or end a relationship, but if adherence to GSZ can, it does so with low probability because I have not run into it. And recently I met someone in a book store and since we were standing next to books on ethics while I was introducing myself to her, I gave a brief introduction to my ethical ideas along with the URL of my web pages on ethics. Then she contacted me, thanked me for causing her to discover Eliezer's page on the 12 virtues of rationality, and agreed to go on a date with me.
I would be willing to go into more detail if you want to know more.
Richard:
Richard: "Is it not true for example that it increases the probability of the survival of my loved one for me to create billions of copies of her and scatter them as widely as possible? And of course each copy should be surrounded by an armada of starships dedicated to her protection."
Desiring to maximize the survival probability of a person by copying them millions of times is NOT the usual way that humans love one another. That you suggest this worries me, because it indicates that when you say "love" you mean something rather different than I do. Go out and ask some people whether they would consider doing that for someone they loved as a token of affection.
Richard: "Moreover, to obtain the important and massive benefits of having the love of a good woman, ..."
You have reduced every aspect of human life to its instrumental value. Richard, you have gone badly, badly astray.
OK, Roko.
Have you made sure she knows her love made you completely re-evaluate the meaning of life?
Since you are letting us know, it seems to me only fair that you make sure she knows.
Have you made sure she knows her love made you completely re-evaluate the meaning of life?
Since you are letting us know, it seems to me only fair that you make sure she knows.
- for various reasons, she is not exactly my favourite person in the world at the moment. And perhaps I overemphasized; I think that the things that happened between us were perhaps the final straw. But I'm sure I will in the fulness of time.
Evolution of a (trans-species) meta-ethics may well arise from the existence of "selfish virtues."
All of us, in our lives, have been helped and/or inspired by individuals who have gone beyond narrow self-interest to form an extended self. These extended selves -- or "souls," for the religious -- manifest and spread universal qualities of spirit.
What if such ways of being have evolved to advance "selfish virtues" - qualities of spirit that seek reproductive success?
E.O. Wilson and Richard Dawkins have argued - correctly I believe - that selfish genes and selfish memes predispose us to make unselfish sacrifices to ensure their reproductive success. They have pointed to a consilience in which selfish genes and selfish memes interact and evolve for mutual benefit.
What if there are three, rather than two, streams in the co-evolutionary spiral of consilience?
Selfish virtues -- in other words, selfish qualities of spirit -- may be the reason that sentient beings sacrifice for others who may have not have _any_ close genetic or memetic affinity.
I believe that our acts of caring and "altruism," in these cases, aim to ensure the preservation and reproductive success of shared qualities of spirit.
This could explains much of the attachment we feel for noble spirits across traditional cultural, national, and other divides -- and even explains the depth of attachment we feel when witnessing acts of kindness and courage in other species.
It may be time, accordingly, to recognize "selfish virtues" as fundamental players in the enchanted circle of consilience, as equal players with genes and memes in evolution's spiral.
If so, what is the best name we can give them, along the lines of genes and memes? Possible names include spires (for qualities of spirit), vals or vales (for values), or fenes (which DK Matai has suggested, for "freedom enhancing narrative envelopes.")
Mark Frazier
Openworld, Inc.
www.openworld.com
@openworld (twitter)
Post a Comment