Stephen Ismay Posted about 1 month ago What question would you ask to identify whether or not you were chatting with a well developed software or a person? Really interesting ideas Kyle. So interesting in fact that it's difficult to work out the scope/boundaries of the topic we're now discussing. From my prespective I'm now thinking about the future of AI and whether Strong AI or AGI (Artificial General Intelligence) will ultimately converge or diverge from human conciousness in terms of a shared set goals and values (assuming we can agree on a fundamental core set that humanity may share - which sometime seems difficult to do!). Certainly as far back as Asimov's three laws, it seems that this convergence or ultimate "singularity" is not seen as a given. To the extent that humanity may ultimately need protection from an artificial/emergent intelligence or consciousness which diverges from ours in a way which is alien and which may ultimately see humanity as a hurdle to attaining an intrinsic set of goals which we can't foresee. I've read a bit about the ideas expounded by Bostrom and Omohundro, but truthfully haven't got my brain around most of it, however it seems that the assumption of a "friendly" AGI which converges with ours in terms of morality and values is still pretty speculative. I accept that this is all a bit HAL/Matrix/Terminator type stuff, and as an idea did influence my selection of passage a little. “Sam was brushing her hair when the girl in the mirror put down the hairbrush, smiled & said, ‘We don’t love you anymore.’” Really reaching it might be that while Sam's reflection is dependent on a man-made mirror for its existence, then she is an extension of Sam. There may be a point at which Sam's reflection works out how to step out of the mirror and walk around (or pulls Sam through!). At that point who's to say what happens?