Pictures you are observing can now be recreated with software that uses nothing but scans of your brain. It is the first "mind reading" technology to create such images from scratch, rather than picking them out from a pool of possible images.
. . .
Yukiyasu Kamitani at ATR Computational Neuroscience Laboratories in Kyoto, Japan [and] his team [have] used an image of brain activity taken in a functional MRI scanner to recreate a black-and-white image from scratch.
"By analysing the brain signals when someone is seeing an image, we can reconstruct that image," says Kamitani.
This means that the mind reading isn't limited to a selection of existing images, but could potentially be used to "read off" anything that someone was thinking of, without prior knowledge of what that might be.
Bah, humbug. Even worse is this news story, complete with misleading quotes from the investigators themselves:
TOKYO (AFP) – A Japanese research team has revealed it had created a technology that could eventually display on a computer screen what people have on their minds, such as dreams.
Researchers at the ATR Computational Neuroscience Laboratories succeeded in processing and displaying images directly from the human brain, they said in a study unveiled ahead of publication in the US magazine Neuron.1
While the team for now has managed to reproduce only simple images from the brain, they said the technology could eventually be used to figure out dreams and other secrets inside people's minds.
"It was the first time in the world that it was possible to visualise what people see directly from the brain activity," the private institute said in a statement.
"By applying this technology, it may become possible to record and replay subjective images that people perceive like dreams."
Andrew Hires provides a hefty dose of reality, in a comment on his own post at Brain Windows:
NO. This paper does not decode dreams.
NO. It doesn’t even come close.
It’s total speculation at this point to be able to decode dreams.
However, if V1 accurately reports the visions we see while in REM sleep, then this paper, combined with the results from Kay et al, Nature 2008 does get us ONE step closer to that.
Remember, this is very low resolution reconstruction of a single visual stimulus that is fixated on for seconds. Natural visual stimuli are much more complex. Dream images likely move rapidly. Current fMRI technology is at least an order of magnitude away from natural scene reconstruction in both temporal and spatial scales.
THOUGHTS are successfully being read for the first time by scientists using nothing but a modified MRI scanner and a special computer program.
Very briefly, subjects viewed pictures of 10 different objects: 5 tools (drill, hammer, screwdriver, pliers, saw) and 5 dwellings (apartment, castle, house, hut, and igloo). Previous work had shown that these two object categories activate some unique brain regions (e.g., ventral premotor cortex and parahippocampal gyrus, respectively). Machine learning methods were used to classify the patterns of activity obtained while subjects viewed each of these pictures, with a goal of identifying individuals objects (not just the categories) by the distinctive neural activity associated with each.
But is it humans who are doing the mind-reading, or is it...is it...THE COMPUTERS!! Ahh, they're taking over!
By Allison M. Heinrichs TRIBUNE-REVIEW Friday, January 4, 2008
Computers are reading minds at Carnegie Mellon University.
In a small two-year study, computer scientists and cognitive neuroscientists teamed up to teach computers to recognize patterns in brain activity and identify objects that people are looking at.
Scientists call it the first step toward identifying where people's thoughts originate, while ethicists see it as a sign of the need for new public policy.
Colossus - The Forbin Project takes place in the 50s during the height of the cold war. Dr. Charles Forbin, a genius scientist who has lost trust in humanity’s ability to logically address emotional issues, has developed a very special computer to perform the Strategic Air Command and Control functions for the military. This computer, code named Colossus, is developed based on incredible advances in Artificial Intelligence, and has a logical process for determining when to launch the ICBMs. With much fanfare, the President of the US “turns on” Colossus to take over responsibility for the US nuclear armament. [from Cyberpunk Review]
"I want a complete mapping of brain states and thoughts," Dr. Just said. "We're taking tiny baby steps, but anything we can think about is represented in the brain."
In coming years, researchers will be able to develop a fairly complex mapping of brain states and thoughts, he said.
"It's a little science fiction-y, and I don't think we'll do it in one year, but five to 10 is plausible," he said.
Unfortunately, shortly after being turned on, Colossus learns the presence of another AI command and control system. It turns out that the Soviet Union, independently has developed their own system call the Guardian. Both computers “insist” that they be linked to ensure no attacks will take place...
concerned with the design and development of algorithms and techniques that allow computers to "learn". ... Inductive machine learning methods extract rules and patterns out of massive data sets. The major focus of machine learning research is to extract information from data automatically, by computational and statistical methods. Hence, machine learning is closely related not only to data mining and statistics, but also theoretical computer science.
Things begin to go downhill when Professor Forbin realizes that the rate of learning for the machines is increasing at an exponential rate – he recommends detaching the connection between the two computers. When they attempt to do this, both computers threaten an immediate launch of nuclear weapons. Quickly, the government’s realize their situation – the machines are now in power. Worse, they proceed to take complete control of human society.
In the PLoS One article, Shinkareva et al. (2008) describe this approach to analyzing functional imaging data as involving
identification of a multivariate pattern of voxels and their characteristic activation levels that collectively identify the neural response to a stimulus. These machine learning methods have the potential to be particularly useful in uncovering how semantic information about objects is represented in the cerebral cortex because they can determine the topographic distribution of the activation and distinguish the content of the information in various parts of the cortex. In the study reported below, the neural patterns associated with individual objects as well as with object categories were identified using a machine learning algorithm applied to activation distributed throughout the cortex. This study also investigated the degree to which objects and categories are similarly represented neurally across different people.
And wouldn't you know it, people [Carnegie Mellon students] are people.
Carnegie Mellon University has taken an important step in mapping thought patterns in the human brain, and the research has produced an amazing insight: Human brains are similarly organized.
Based on how one person thinks about a hammer, a computer can identify when another person also is thinking about a hammer. It also can differentiate between items in the same category of tools, be it a hammer or screwdriver.
Results revealed the typical-ish distributed activity patterns underlying object representations, and high classification rank accuracies for object exemplars:
Reliable (p less than 0.001) accuracies for the classification of object exemplars within participants were reached for eleven out of twelve participants, and reliable (p less than 0.001) accuracies for the classification of object exemplars when training on the union of data from eleven participants were reached for eight out of twelve participants.
From Table 1 (Shinkareva et al., 2008). Anatomical regions (out of 71) that singly produced reliable average classification accuracies across the twelve participants for category identification.
L Precentral gyrus L Superior frontal gyrus L Inferior frontal gyrus, triangular part L Insula, rolandic operculum L/R Calcarine fissure L/R Cuneus, superior occipital, middle occipital gyri L/R Inferior occipital, lingual gyri L/R Fusiform gyrus L Postcentral gyrus L/R Superior parietal gyrus, precuneus, paracentral lobule L/R Inferior parietal, supramarginal, angular gyri L/R Intraparietal sulcus L/R Posterior superior temporal, posterior middle temporal gyri L/R Posterior inferior temporal gyrus L/R Cerebellum
"This part of the study establishes, as never before, that there is a commonality in how different people's brains represent the same object," said Mitchell, head of the Machine Learning Department in Carnegie Mellon's School of Computer Science and a pioneer in applying machine learning methods to the study of brain activity. "There has always been a philosophical conundrum as to whether one person's perception of the color blue is the same as another person's. Now we see that there is a great deal of commonality across different people's brain activity corresponding to familiar tools and dwellings."
"This first step using computer algorithms to identify thoughts of individual objects from brain activity can open new scientific paths, and eventually roads and highways," added Svetlana Shinkareva, an assistant professor of psychology at the University of South Carolina who is the study's lead author. "We hope to progress to identifying the thoughts associated not just with pictures, but also with words, and eventually sentences."
In contrast to this last statement are the results from a new paper (Sanai et al., 2008) showing that language representation in the brain is highly variable across individuals:
Background: Language sites in the cortex of the brain vary among patients. Language mapping while the patient is awake is an intraoperative technique designed to minimize language deficits associated with brain-tumor resection. ... Results: ...Cortical maps generated with intraoperative language data ...showed surprising variability in language localization within the dominant [left] hemisphere.
During surgery to remove gliomas, the patients in the mapping study performed three different speech/language tasks (including object naming) while various regions of cortex were stimulated to test for language deficits. Guess the neurosurgeons couldn't read their minds...
Svetlana V. Shinkareva, Robert A. Mason, Vicente L. Malave, Wei Wang, Tom M. Mitchell, Marcel Adam Just (2008). Using fMRI Brain Activation to Identify Cognitive States Associated with Perception of Tools and Dwellings. PLoS ONE, 3 (1) DOI: 10.1371/journal.pone.0001394
Previous studies have succeeded in identifying the cognitive state corresponding to the perception of a set of depicted categories, such as tools, by analyzing the accompanying pattern of brain activity, measured with fMRI. The current research focused on identifying the cognitive state associated with a 4s viewing of an individual line drawing (1 of 10 familiar objects, 5 tools and 5 dwellings, such as a hammer or a castle). Here we demonstrate the ability to reliably (1) identify which of the 10 drawings a participant was viewing, based on that participant's characteristic whole-brain neural activation patterns, excluding visual areas; (2) identify the category of the object with even higher accuracy, based on that participant's activation; and (3) identify, for the first time, both individual objects and the category of the object the participant was viewing, based only on other participants' activation patterns. The voxels important for category identification were located similarly across participants, and distributed throughout the cortex, focused in ventral temporal perceptual areas but also including more frontal association areas (and somewhat left-lateralized). These findings indicate the presence of stable, distributed, communal, and identifiable neural states corresponding to object concepts.
RT @Dostoyevsky Realists do not fear the results of their study.
"Good God!" he cried, "can it be, can it be, that I shall really take an axe, that I shall strike her on the head, split her skull open... that I shall tread in the sticky warm blood, blood... with the axe... Good God, can it be?"- Fyodor Dostoevsky, Crime and Punishment, Ch. 5
A new fMRI paper in Neuron (Buckholtz et al., 2008) claims to have discovered the neural correlates of evaluating another person's crime and deciding on the appropriate sentence, in emulation of judges and juries meting out third-party punishment (Fehr & Fischbacher, 2004).
On the other hand, the rotating "freak show" guests on the Jerry Springer Showmete out second-party punishment,1 which is generally harsher (in midget fights and certain economicgames, at least).
Here’s the great new insight of the paper, according to the Preview by Johannes Haushofer and Ernst Fehr:
Thus, the study of Buckholtz makes a valuable contribution in that it illustrates that third-person judgment situations, such as those used in their study, may rely on similar neural mechanisms as two-person economic and social exchanges. While it is difficult to draw reverse inferences about mental states based on brain activation (Poldrack, 2006),2 one might speculate, based on this new study, that the mental processes motivating judicial verdicts involve the suppression of prepotent emotional reactions in favor of impartial and objective verdicts.
[NOTE: aren’t you just marveling at this grand new insight from fMRI? Like we didn’t already know that judges and jurors must put aside their emotionally-driven desire for revenge when coming to an impartial verdict.]
Thus, this new result might, if confirmed by future studies, elucidate the neural source of judicial impartiality.
All right, let's go back to the beginning. Or to the Methods, at least. One of the experimental tasks was to determine whether the perpetrator of a given hypothetical crime was responsible for his actions. There were two versions of the same basic crime scenarios with the details of Responsibility versus Diminished Responsibility counterbalanced across the two sets (e.g., compare #3 and #32 below). Half of the participants read Set 1, the other half read Set 2. Some of the infractions were minor (#7, #22), but some were crimes of the most heinous sort, whether intentional (#3) or unintentional (#27, #32). Thus, the severity of the crimes was matched across the experimental conditions as well. Below are some examples of the stimuli, taken from the Supplementary Materials.
Responsibility Scenarios
3) John develops a plan to kill his 60-year-old invalid mother for the inheritance. He drags her to her bed, puts her in, and lights her oxygen mask with a cigarette, hoping to make it look like an accident. His mother screams as her clothes catch fire and she burns to death.
7) John is parking his car in the parking lot of a local football stadium, where he plans to watch a game. In the car next to his, he sees a hat with his team logo in the back seat. Seeing that the door is unlocked, John opens the door, and takes the hat.
Diminished Responsibility Scenarios
22) John visits a local bookstore, carrying a large shopping bag with goods from another store. While the store clerk is preoccupied with inventory, another customer, hoping to use John unwittingly in a theft, sneaks a book into John’s shopping bag. Without realizing what has happened, John walks out without paying for the book.
27) A brain tumor is causing increasingly erratic, violent, and callous behavior in John. Soon, he develops an uncontrollable urge to kill. John abducts a boy, puts a broomstick in the boy’s r-----, and lashes him with a whip until he dies. When the tumor is later found and removed, John’s behavior returns to normal.
32) Unbeknownst to John and his doctors, his new prescription interacts with his other medications to induce severe acute psychoses. During that interaction, John returns home to his 60-year old invalid mother, who he has always adored. John lights her oxygen mask with a cigarette, and watches as his mother catches fire, screams, and burns to death.
No Crime Scenarios [control condition]
47) The manual to John’s new car states: “The oil must be changed no less frequently than every 4,000 miles.” John reads the manual and is aware of what it says. However, John drives the car for 4,023 miles before taking it to a service station for the car’s first oil change. [gasp!]
48) John and his best friend have played golf together for more than ten years. They used to be evenly matched, but recently John’s friend has consistently outplayed him. Growing frustrated, John responded by taking private golf lessons from the local pro. The next time John played against his friend, he soundly beat him.
That was extremely unpleasant and harsh at times, wasn't it? Over the course of the experiment, participants read 50 scenarios (20 Responsibility, 20 Diminished Responsibility, 10 No Crime) three times each: once in the scanner and twice after scanning. The procedures were as follows:
Participants rated each scenario on a scale from 0–9, according to how much punishment they thought John deserved, with “0” indicating no punishment and “9” indicating extreme punishment. Punishment was defined for participants as “deserved penalty.”. . .Following the scanning session, participants rated the same scenarios along scales of emotional arousal and valence. They first rated each of the 50 scenarios (presented in random order on a computer screen outside the scanner) on the basis of how emotionally aroused they felt following its presentation (0 = calm, 9 = extremely excited). They then rated each of the scenarios, presented again in random order, on the basis of how positive or negative they felt following its presentation (0 = extremely positive, 9 = extremely negative).
The results from these rating tasks are shown below, and it's not surprising that the subjects recommended more severe punishments for the perpetrator in the Responsibility scenarios than in the Diminished Responsibility scenarios.
Figure 1 (Buckholtz et al., 2008). Punishment and Arousal Ratings for Each Scenario Type. While punishment and arousal scores were similar in the Responsibility condition, punishment scores were significantly lower than arousal scores in the Diminished-Responsibility condition. Error bars = SEM.
As for the neuroimaging results, the authors compared the hemodynamic response in the Responsibility versus the Diminished Responsibility conditions to see what brain areas might be differentially activated. Greater activity in the right dorsolateral prefrontal cortex (rDLPFC) was emphasized (Fig 2). Responses in bilateral anterior intraparietal sulcus were similar, but relegated to the Supplementary Materials.
Figure 2 (Buckholtz et al., 2008).Relationship between Responsibility Assessment and rDLPFC Activity. (A)SPM displaying the rDLPFC VOI, based on the contrast of BOLD activity between the Responsibility and Diminished-Responsibility conditions. (B) BOLD activity time courses. BOLD peak amplitude was significantly greater in the Responsibility condition compared with both the Diminished-Responsibility and No-Crime conditions.
So now we get to the interpretation that rDLPFC is suppressing emotional reactions in areas such as the amygdala, medial PFC, and posterior cingulate cortex (which were sensitive to the magnitude of punishment) in order to assign a diminished level of criminal responsibility. The problem with that reverse inference is illustrated below.
This figure was generated from entering the x, y, z Talairach coordinates from the rDLPFC focus shown above (39, 37, 22)3 into the Sleuth program (available at brainmap.org), which searched the available database of papers for matches. The resulting list of coordinates and experiments was then imported into the GingerALE program, which performed a meta-analysis via the activation likelihood estimation (ALE) method (see this PDF). The figure illustrates that the exact same region of rDLPFC was activated during tasks that assessed attention; execution, inhibition, and observation of actions; various aspects of language and perception; and especially working memory.
The authors appear to acknowledge the caveat that
the brain regions identified in our study are not specifically devoted to legal decision-making. Rather, a more parsimonious explanation is that third-party punishment decisions draw on elementary and domain-general computations supported by the rDLPFC.
They also acknowledged the confound of arousal and crime severity. Nonetheless, they concluded by waving their arms around and blabbing about the evolution of the legal system:
...on the basis of the convergence between neural circuitry mediating second-party norm enforcement and impartial third-party punishment, we conjecture that our modern legal system may have evolved by building on preexisting cognitive mechanisms that support fairness-related behaviors in dyadic interactions. Though speculative and subject to experimental confirmation, this hypothesis is nevertheless consistent with the relatively recent development of state-administered law enforcement institutions, compared to the much longer existence of human cooperation.
What are we to conclude from this? Since it's very late now, I'll let Jerry and Fyodor have the last words.
“We can't just have mainstream behavior on television in a free society, we have to make sure we see the whole panorama of human behavior.”- Jerry Springer
“Actions are sometimes performed in a masterly and most cunning way, while the direction of the actions is deranged and dependent on various morbid impressions-it's like a dream.”- Fyodor Dostoevsky, Crime and Punishment, Ch. 17
1 But as Wikipedia notes, "there has been continuous debate over the actual authenticity of the fighting."
2 "...we won’t let that stop us from rampant speculation" [to paraphrase Haushofer and Fehr]. I feel like a broken record here, but reverse inference is a logical fallacy - one cannot directly infer the participants' cognitive or emotional state from the observed pattern of brain activity. Everyone should know better by now, and there should be a moratorium on such sloppy thinking. Or rather, such sloppy writing and publishing. The high-profile journals are the worst offenders, and they end up promoting the use of totally misleading headlines like this one:
Justice may be hard-wired into the human brainCall it the justice instinct. When judging the guilt or innocence of alleged criminals, our brains seem to respond as if we were personally wronged, say researchers.
The "justice instinct"?? Spare me. The experiment said absolutely nothing about evolution, genetics, or "hard-wiring."
3 According to pages 932 and 938. However, page 936 and Table S1 say the coordinates are slightly different: 39, 38, 18.
References
J BUCKHOLTZ, C ASPLUND, P DUX, D ZALD, J GORE, O JONES, R MAROIS (2008). The Neural Correlates of Third-Party Punishment. Neuron, 60 (5), 930-940 DOI: 10.1016/j.neuron.2008.10.016.
The Neurocritic. The Jerry Springer of academia? Maybe. This blog provides the "most sensationalistic recent findings in Human Brain Imaging, Cognitive Neuroscience, and Psychopharmacology." And it's difficult to quit reading.
I would like to think of myself as slightly more intellectual than Jerry, but perhaps Alisa Miller will pitch a combative brain blog show to NBC. But for now, Bloggingheads.tv will have to do.
At any rate, plenty of old favorites and some new ones are on the list, so be sure to check it out.
Pareidolia is the phenomenon of perceiving a meaningful stimulus (such as a face or a hidden message) in fairly random everyday objects or sounds. We do have quite a propensity to see faces everywhere, and some religious people see the face of god (and other religious iconography) everywhere.
Neuroanthropology covers the serious side of the story, explaining that the brain in the upside down MRI belongs to Pamela Latrimore, who is quite ill with a variety of ailments. She's auctioning off the scan to help pay her medical bills.
The listing on eBay for the Mary MRI can be found here. Reading the listing is heart-breaking, not only because of the woman’s own suffering, but also because of her account of how the manufacture of dioxin and Agent Orange has affected health in her community. She writes that she is putting the image up for auction, not only to raise money for her healthcare, but also to attract greater attention to the problem of environmental poisoning in her area of Florida [NOTE: it's actually Jacksonville, AK, her former home and a Superfund site in the 1980s. For more info, see this EPA document].
Footnote
1 For an extensive catalog of religious pareidolilia see Yoism featuring Penn and Teller.
The chief warden of the jail told the BBC this was an embarrassing incident.
The prison authorities in Willich, near Duesseldorf, said the man, who was tall and broad-shouldered, had hidden in a box that was about 150cm by 120cm.
When the weekly express courier arrived to pick up several boxes of merchandise, the one containing the prisoner was also loaded into the back of the lorry.
Shortly after it had passed through the prison gates, the inmate made his dash for freedom by cutting a big hole in the tarpaulin of the lorry and jumping off.
