Top tips for new experimenters
April 9th, 2012 § 5 Comments
I set out to write my top five tips for new experimenters today and found there were really only two universal suggestions I felt I could and should make:
- Simplify your design. There is no complex question that can’t be better asked with a simple one. Simple design means stronger statistics, a clearer interpretation, and less variables to control. If you cannot phrase your core question in a sentence, you need to drastically reduce the scope of your experiment.
- Know your design. Before collecting the data, you should know exactly what kind of data it will be, how many variables, and what kind of statistical test you will use to analyze it. Then you need to collect 4-5 “throw-away” participants and run them through this analysis. This ensures that the data can be readily analyzed in a rigorous way, from start to finish. You will know you are ready when your looking at the successful results of a pilot, which will discover study-killing bugs (of which there are MANY)
Good post. I would add a #3, it’s kind of an aspect of #2 although important enough to stand alone -
Make sure you are one of the pilot subjects. It’s amazing what kind of things you notice when you’re actually in the scanner that you never otherwise would – anything from the fact that the stimuli aren’t very visible, to the fact that the sequence you’re using makes the bed shake, to the fact that the task is just so long & boring that you fall asleep by the end (which is so much easier in the scanner than when you’re sitting up at a computer, which is when you probably piloted the task!)
If you’re not MRI safe, get a trusted fellow researcher to do it. But never assume that non-scientist volunteers will tell you these things because they don’t (I think because they don’t want to look stupid by questioning your authority.)
Insula and Anterior Cingulate: the ‘everything’ network or systemic neurovascular confound?
February 18th, 2012 § 14 Comments
It’s no secret in cognitive neuroscience that some brain regions garner more attention than others. Particularly in fMRI research, we’re all too familiar with certain regions that seem to pop up in study after study, regardless of experimental paradigm. When it comes to areas like the anterior cingulate cortex (ACC) and insula (AIC), the trend is obvious. Generally when I see the same brain region involved in a wide a variety of tasks, I think there must be some very general level function which encompasses these paradigms. Off the top of my head, the ACC and AIC are major players in cognitive control, pain, emotion, consciousness, salience, working memory, decision making, and interoception to name a few. Maybe on a bad day I’ll look at a list like that and think, well localization is just all wrong, and really what we have is a big fat prefrontal cortex doing everything in conjunction. A paper published yesterday in Cerebral Cortex took my breath away and lead to a third, more sinister option: a serious methodological confound in a large majority of published fMRI papers.
Neurovascular coupling and the BOLD signal: a match not made in heaven
An important line of research in neuroimaging focuses on noise in fMRI signals. The essential problem of fMRI is that, while it provides decent spatial resolution, the data is acquired slowly and indirectly via the blood-oxygenation level dependent (BOLD) signal. The BOLD signal is messy, slow, and extremely complex in its origins. Although we typically assume increasing BOLD signal equals greater neural activity, the details of just what kind of activity (e.g. excitatory vs inhibitory, post-synaptic vs local field) are murky at best. Advancements in multi-modal and optogenetic imaging hold a great deal of promise regarding the signal’s true nature, but sadly we are currently at a “best guess” level of understanding. This weakness means that without careful experimental design, it can be difficult to rule out non-neural contributors to our fMRI signal. Setting aside the worry about what neural activity IS measured by BOLD signal, there is still the very real threat of non-neural sources like respiration and cardiovascular function confounding the final result. This is a whole field of research in itself, and is far too complex to summarize here in its entirety. The basic issue is quite simple though.
End-tidal C02, respiration, and the BOLD Signal
In a nutshell, the BOLD signal is thought to measure downstream changes in cerebral blood-flow (CBF) in response to neural activity. This relationship, between neural firing and blood flow, is called neurovascular coupling and is extremely complex, involving astrocytes and multiple chemical pathways. Additionally, it’s quite slow: typically one observes a 3-5 second delay between stimulation and BOLD response. This creates our first noise-related issue; the time between each ‘slice’ of the brain, or repetition time (TR), must be optimized to detect signals at this frequency. This means we sample from our participant’s brain slowly. Typically we sample every 3-5 seconds and construct our paradigms in ways that respect the natural time lag of the BOLD signal. Stimulate too fast, and the vasculature doesn’t have time to respond. Stimulation frequency also helps prevent our first simple confound: our pulse and respiration rates tend oscillate at slightly slower frequencies (approximately every 10-15 seconds). This is a good thing, and it means that so long as your design is well controlled (i.e. your events are properly staggered and your baseline is well defined) you shouldn’t have to worry too much about confounds. But that’s our first problematic assumption; consider for example when one’s paradigms use long blocks of obscure things like “decide how much you identify with these stimuli”. If cognitive load differs between conditions, or your groups (for example, a PTSD and a control group) react differently to the stimuli, respiration and pulse rates might easily begin to overlap your sampling frequency, confounding the BOLD signal.
But you say, my experiment is well controlled, and there’s no way my groups are breathing THAT differently! Fair enough, but this leads us to our next problem: end tidal CO2. Without getting into the complex physiology, end-tidal CO2 is a by-product of respiration. When you hold your breath, CO2 blood levels rise dramatically. CO2 is a potent vasodilator, meaning it opens blood vessels and increases local blood flow. You’ve probably guessed where I’m going with this: hold your breath in the fMRI and you get massive alterations in the BOLD signal. Your participants don’t even need to match the sampling frequency of the paradigm to confound the BOLD; they simply need to breath at slightly different rates in each group or condition and suddenly your results are full of CO2 driven false positives! This is a serious problem for any kind of unconstrained experimental design, especially those involving poorly conceptualized social tasks or long periods of free activity. Imagine now that certain regions of the brain might respond differently to levels of CO2.
This image is from Change & Glover’s paper, “Relationship between respiration, end-tidal CO2, and BOLD signals in resting-state fMRI”. Here they measure both CO2 and respiration frequency during a standard resting-state scan. The image displays the results of group-level regression of these signals with BOLD. I’ve added circles in blue around the areas that respond the strongest. Without consulting an atlas, we can clearly see that bilateral anterior insula extending upwards into parietal cortex, anterior cingulate, and medial-prefrontal regions are hugely susceptible to respiration and CO2. This is pretty damning for resting-state fMRI, and makes sense given that resting state fluctuations occur at roughly the same rate as respiration. But what about well-controlled event related designs? Might variability in neurovascular coupling cause a similar pattern of response? Here is where Di et al’s paper lends a somewhat terrifying result:
Di et al recently investigated the role of vascular confounds in fMRI by administrating a common digit-symbol substitution task (DSST), a resting state, and a breath-holding paradigm. Signals related to resting-state and breath-holding were then extracted and entered into multiple regression with the DSST-related activations. This allowed Di et al to estimate what brain regions were most influenced by low-frequency fluctuation (ALFF, a common resting state measure) and purely vascular sources (breath-holding). From the figure above, you can see that regions marked with the blue arrow were the most suppressed, meaning the signal explained by the event-related model was significantly correlated with the covariates, and in red where the signal was significantly improved by removal of the covariates. The authors conclude that “(results) indicated that the adjustment tended to suppress activation in regions that were near vessels such as midline cingulate gyrus, bilateral anterior insula, and posterior cerebellum.” It seems that indeed, our old friends the anterior insula and cingulate cortex are extremely susceptible to neurovascular confound.
What does this mean for cognitive neuroscience? For one, it should be clear that even well-controlled fMRI designs can exhibit such confounds. This doesn’t mean we should throw the baby out with the bathwater though; some designs are better than others. Thankfully it’s pretty easy to measure respiration with most scanners, and so it is probably a good idea at minimum to check if one’s experimental conditions do indeed create differential respiration patterns. Further, we need to be especially cautious in cases like meditation or clinical fMRI, where special participant groups may have different baseline respiration rates or stronger parasympathetic responses to stimuli. Sadly, I’m afraid that looking back, these findings greatly limit our conclusions in any design that did not control for these issues. Remember that the insula and ACC are currently cognitive neuroscience’s hottest regions. I’m not even going to get into resting state, where these problems are all magnified 10 fold. I’ll leave you with this image from neuroskeptic, estimating the year’s most popular brain regions:
Are those spikes publication fads, every-task regions, or neurovascular artifacts? You be the judge.
edit:As many of you had questions or comments regarding the best way to deal with respiratory related issues, I spoke briefly with resident noise expert Torben Lund at yesterday’s lab meeting. Removal of respiratory noise is fairly simple, but the real problem is with end-tidal C02. According to Torben, most noise experts agree that regression techniques only partially remove the artifact, and that an unknown amount is left behind even following signal regression. This may be due to slow vascular saturation effects that build up and remain irrespective of shear breath frequency. A very tricky problem indeed, and certainly worth researching. Note: credit goes to my methods teacher and fMRI noise expert Torben Lund, and CFIN neurobiologist Rasmus Aamand, for introducing and explaining the basis of the C02/respiration issue to me. Rasmus particularly, whose sharp comments lead to my including respiration and pulse measures in my last meditation project.Daniel Wolpert on TED: Noise reduction and the Bayesian Brain
February 9th, 2012 § Leave a Comment
This has to be the best introduction to the Bayesian Brain hypothesis I have ever seen. I’m normally somewhat cautious about my endorsement of this view, but Daniel really draws you in. Beautiful talk, very understandable and yet not too dumbed-down.
Zombies on the Brain Redux: Return of the Baroness
August 15th, 2011 § 1 Comment
Just a quick note. As most of you are probably aware, in the past week a bit of a furor has erupted on the interwebs regarding a certain UK Noblewoman’s concerns regarding our free time, and it’s impact on our brain. Susan Greenfield has written extensively, if without much depth, on the need to sound the alarm regarding how much time we spend staring at a screen. Thank goodness the kind, brillant minds have responded, and they’ve done so negatively. I’m not going to reproduce the debate here, but it suffices to say Susan’s now been shredded publicly by both world-leading Autism experts and folks who actually research our digital habits.Anyway, this is just a short post to share my view on the whole thing:
It’s not that i’m not sympathetic to the idea we spend too much time in front of a screen. It’s levying the weight of that argument around some BS neurobabble. The brain, and it’s astounding capacity for change, need not enter this debate in an alarmist fashion. If we want to discuss how our society spends its time, let’s not cling onto poorly understood scientific phenomena to do so. Let’s talk about radical capitalism and the 40 hour work week. I spend 38+ hours a week online, programming, and generally living on a computer as do many contemporary workers. Talking about the brain won’t change that- but it will sell books. And I think it puts a clear perspective on the 3-4 hours a week I might spend playing games. The problem isn’t brains, games, or anything in between. Anyone who tries to tell you otherwise wants to make a fast buck off you.
Susan is hoping to use her high-standing to kick off some kind of alarmist movement akin to the global warming debate. She’s even been so cheeky as to give her cause a similar sounding name “Mindchange”, so that her would be activists might have some nice banner around which to rally. Plus, it makes for some pretty obvious best-selling book titles. Maybe even a series of self-help audiobooks filled to the brim with half-truisms, folksy inspiration, and plenty of badly misunderstood science.
And that’s just the problem. Beneath the seemingly innocent wish to make a legacy and buck for herself, Susan has denigrated one big problem (global warming), disrespected the science, and mis-educated her audience. Not only that but I believe she’s obscuring the real problem. Call me radical, but if we don’t start to take serious the cultural-systemic problems that threaten our world, we’re fucked. By capitalizing on sensation, the Baroness has obscured a legitimate debate about the way we spend our lives.
2010 in review
January 2nd, 2011 § Leave a Comment
The stats helper monkeys at WordPress.com mulled over how this blog did in 2010, and here’s a high level summary of its overall blog health:

The Blog-Health-o-Meter™ reads This blog is on fire!.
Crunchy numbers
A Boeing 747-400 passenger jet can hold 416 passengers. This blog was viewed about 4,400 times in 2010. That’s about 11 full 747s.
In 2010, there were 13 new posts, growing the total archive of this blog to 17 posts. There were 13 pictures uploaded, taking up a total of 2mb. That’s about a picture per month.
The busiest day of the year was July 1st with 182 views. The most popular post that day was My response to Carr and Pinker on Media Plasticity.
Where did they come from?
The top referring sites in 2010 were neuroskeptic.blogspot.com, twitter.com, healthfitnesstherapy.com, facebook.com, and Google Reader.
Some visitors came searching, mostly for neuroconscience, micah allen, inferior parietal lobule, dlpfc, and neurological basis of development.
Attractions in 2010
These are the posts and pages that got the most views in 2010.
My response to Carr and Pinker on Media Plasticity June 2010
4 comments
Snorkeling ’the shallows’: what’s the cognitive trade-off in internet behavior? June 2010
9 comments and 2 Likes on WordPress.com
The Author October 2009
1 comment
Brain Plasticity, Distributed Social Cognition, and the Luddite Notion. February 2010
5 comments
Switching between executive and default mode networks in posttraumatic stress disorder [excerpts and notes] August 2010
Snorkeling ’the shallows’: what’s the cognitive trade-off in internet behavior?
June 8th, 2010 § 12 Comments
I am quite eager to comment on the recent explosion of e-commentary regarding Nicolas Carr’s new book. Bloggers have already done an excellent job summarizing the response to Carr’s argument. Further, Clay Shirky and Jonah Lehrer have both argued convincingly that there’s not much new about this sort of reasoning. I’ve also argued along these lines, using the example of language itself as a radical departure from pre-linguistic living. Did our predecessors worry about their brains as they learned to represent the world with odd noises and symbols?
Surely they did not. And yet we can also be sure that the brain underwent a massive revolution following the acquisition of language. Chomsky’s linguistics would of course obscure this fact, preferring us to believe that our linguistic abilities are the amalgation of things we already possessed: vision, problem solving, auditory and acoustic control. I’m not going to spend too much time arguing against the modularist view of cognition however; chances are if you are here reading this, you are already pretty convinced that the brain changes in response to cultural adaptations.
It is worth sketching out a stock Chomskyian response however. Strict nativists, like Chomsky, hold that our language abilities are the product of an innate grammar module. Although typically agnostic about the exact source of this module (it could have been a genetic mutation for example), nativists argue that plasticity of the brain has no potential other than slightly enhancing or decreasing our existing abilities. You get a language module, a cognition module, and so on, and you don’t have much choice as to how you use that schema or what it does. The development of anguage on this view wasn’t something radically new that changed the brain of its users but rather a novel adaptation of things we already and still have.
To drive home the point, it’s not suprising that notable nativist Stephen Pinker is quoted as simply not buying the ‘changing our brains’ hypothesis:
“As someone who believes both in human nature and in timeless standards of logic and evidence, I’m skeptical of the common claim that the Internet is changing the way we think. Electronic media aren’t going to revamp the brain’s mechanisms of information processing, nor will they supersede modus ponens or Bayes’ theorem. Claims that the Internet is changing human thought are propelled by a number of forces: the pressure on pundits to announce that this or that “changes everything”; a superficial conception of what “thinking” is that conflates content with process; the neophobic mindset that “if young people do something that I don’t do, the culture is declining.” But I don’t think the claims stand up to scrutiny.”
Pinker makes some good points- I agree that a lot of hype is driven by the kinds of thinking he mentions. Yet, I do not at all agree that electronic media cannot and will not revamp our mechanisms for information processing. In contrast to the nativist account, I think we’ve better reason than ever to suspect that the relation between brain and cognition is not 1:1 but rather dynamic, evolving with us as we develop new tools that stimulate our brains in unique and interesting ways.
The development of language massively altered the functioning of our brain. Given the ability to represent the world externally, we no longer needed to rely on perceptual mechanisms in the same way. Our ability to discriminate amongst various types of plant, or sounds, is clearly sub-par to that of our non-linguistic brethren. And so we come full circle. The things we do change our brains. And it is the case that our brains are incredibly economical. We know for example that only hours after limb amputation, our somatosensory neurons invade the dormant cells, reassigning them rather than letting them die off. The brain is quite massively plastic- Nicolas Carr certainly gets that much right.
Perhaps the best way to approach this question is with an excerpt from social media. I recently asked of my fellow tweeps,
To which an astute follower replied:
Now, I do realize that this is really the central question in the ‘shallows’ debate. Moving from the basic fact that our brains are quite plastic, we all readily accept that we’re becoming the subject of some very intense stimulation. Most social media, or general internet users, shift rapidly from task to task, tweet to tweet. In my own work flow, I may open dozens and dozens of tabs, searching for that one paper or quote that can propel me to a new insight. Sometimes I get confused and forget what I was doing. Yet none of this interferes at all with my ‘deep thinking’. Eventually I go home and read a fantastic sci-fi book like Snowcrash. My imagination of the book is just as good as ever; and I can’t wait to get online and start discussing it. So where is the trade-off?
So there must be a trade-off, right? Tape a kitten’s eyes shut and its visual cortex is re-assigned to other sensory modalities. The brain is a nasty economist, and if we’re stimulating one new thing we must be losing something old. Yet what did we lose with language? Perhaps we lost some vestigial abilities to sense and smell. Yet we gained the power of the sonnet, the persuasion of rhetoric, the imagination of narrative, the ability to travel to the moon and murder the earth.
In the end, I’m just not sure it’s the right kind of stimulation. We’re not going to lose our ability to read in fact, I think I can make an extremely tight argument against the specific hypothesis that the internet robs us of our ability to deep-think. Deep thinking is itself a controversial topic. What exactly do we mean by it? Am I deep thinking if I spend all day shifting between 9 million tasks? Nicolas Carr says no, but how can he be sure those 9 million tasks are not converging around a central creative point?
I believe, contrary to Carr, that internet and social media surfing is a unique form of self stimulation and expression. By interacting together in the millions through networks like twitter and facebook, we’re building a cognitive apparatus that, like language, does not function entirely within the brain. By increasing access to information and the customizability of that access, we’re ensuring that millions of users have access to all kinds of thought-provoking information. In his book, Carr says things like ‘on the internet, there’s no time for deep thought. it’s go go go’. But that is only one particular usage pattern, and it ignores ample research suggesting that posts online may in fact be more reflective and honest than in-person utterances (I promise, I am going to do a lit review post soon!)
Today’s internet user doesn’t have to conform to whatever Carr thinks is the right kind of deep-thought. Rather, we can ‘skim the shallows’ of twitter and facebook for impressions, interactions, and opinions. When I read a researcher, I no longer have to spend years attending conferences to get a personal feel for them. I can instead look at their wikipedia, read the discussion page, see what’s being said on twitter. In short, skimming the shallows makes me better able to choose the topics I want to investigate deeply, and lets me learn about them in whatever temporal pattern I like. Youtube with a side of wikipedia and blog posts? Yes please. It’s a multi-modal whole brain experience that isn’t likely to conform to ‘on/off’ dichotomies. Sure, something may be sacrificed, but it may not be. It might be that digital technology has enough of the old (language, vision, motivation) plus enough of the new that it just might constitute or bring about radically new forms of cognition. These will undoubtably change or cognitive style, perhaps obsoleting Pinker’s Bayesian mechanisms in favor of new digitally referential ones.
So I don’t have an answer for you yet ToddStark. I do know however, that we’re going to have to take a long hard look at the research review by Carr. Further, it seems quite clear that there can be no one-sided view of digital media. It’s not anymore intrinsically good or bad than language. Language can be used to destroy nations just as it can tell a little girl a thoughtful bed time story. If we’re to quick to make up our minds about what internet-cognition is doing to our plastic little brains, we might miss the forest for the trees. The digital media revolution gives us the chance to learn just what happens in the brain when its’ got a shiny new tool. We don’t know the exact nature of the stimulation, and finding out is going to require a look at all the evidence, for and against. Further, it’s a gross oversimplification to talk about internet behavior as ‘shallow’ or ‘deep’. Research on usage and usability tells us this; there are many ways to use the internet, and some of them probably get us thinking much deeper than others.




