Listening to data

I enjoy browsing /r/dataisbeautiful to see all the creative data visualizations, which makes me wish for more innovative visualization techniques in scientific literature. Unfortunately, anything besides static images is still largely relegated to the supplementary info. Libraries like D3.js are amazing, but the scientific community has yet to embrace these types of tools. This will hopefully change as journals such as eLife become more popular. Until then, I think we’re stuck with static images being the principal method for displaying data. There’s nothing specifically wrong with that, but different approaches can often lead to new insights.

To show you what I mean, I’ve put together a few videos demonstrating a different way of “looking” at time series data: sound synthesis. There are three example below, each using a different data source. I’ll be the first to admit that these are essentially gimmicks, but I hope I can convince you that it’s a useful tool during the data exploration phase, which can lead to new insights and analyses.

Example 1: Neuroscience

I’ll start with something from my own research. I’ve taken a 15 s recording from a neuron in the auditory midbrain and simply used the raw waveform from the signal to create a sound. This signal is the voltage from an extracellular electrode that is slowly advanced through the brain until it a contact is close enough to a particular neuron that we can see individual action potentials. During an experiment, we have the voltage playing continuously through a speaker, since it’s helpful to listen for the distinctive sounds of an action potential when searching for a neuron. Below is an example of we hear when we get close to a neuron.

The action potentials are the large “spikes” in the waveform that are shown in more detail on the smaller set of axes. You can hear them as distinctive crackling sounds. The spikes in this example appear to occur randomly, which we call “spontaneous” spiking since no stimulus is being applied. However, there is a limit imposed by the kinetics of action potential generation on how fast any neuron can fire, often referred to as the refractory period. For this reason, the spiking of a neuron is not truly a Poisson process as it is often assumed to be. You can sometimes even hear a distinct periodicity in the spiking, which is the basis for neuronal oscillations that play critical roles in brain function. These periodicities can be quantified with tools like spectral analysis or directional statistics.

Example 2: Physics/astronomy

This next example is obviously not my field of expertise, but I think it’s nevertheless fascinating. The audio is taken from the Space Audio page from a group at the University of Iowa. It’s a recording of “whistlers”, a phenomenon that can occur when lightning causes radio waves to travel along the Earth’s magnetic field lines, which causes dispersion due to the different speeds at which the different frequencies propagate. Here’s a figure from the same group that explains it better. The result is a distinctive whistling sound where the pitch drops as a result of the dispersion.

The visualization is a simple spectrogram, which shows frequency on the y-axis, time on the x-axis, and intensity as the color. You can see the curved lines in the spectrogram that correspond to the whistling sounds, which is a sign that the frequency is changing over time.

I love this example for two reasons (besides the fact that it just sounds cool). First, the signal did not need to be shifted at all in terms of frequency before converting to sound. You can find examples of “space sounds” where something like gamma radiation is used to synthesize a sound. However, gamma rays have frequencies that are much, much higher than our audible range, so it needs to be shifted in order to hear it. This is largely an arbitrary process, since the perception of the sound can vary depending on how much frequency shifting/compressing is done.

Second, dispersion is a concept important to the propagation of waves in general, so you can find examples of it in the auditory system as well. The cochlea, for example, acts like a dispersive delay line, meaning that it not only separates out the frequency content of sounds, but it does so at slightly different speeds, resulting in what is referred to as the “traveling wave.” These can look a lot like whistlers.

Example 3: The button

Frequent visitors to reddit will probably recall /r/thebutton, reddit’s most recent April Fool’s Day joke. Explaining it fully would take a while, so check out the wiki on the subreddit linked above or this blog post for more information. Suffice it to say, it’s a very interesting dataset. The official server-side data was released after the button ended, and I used it to synthesize sound.

It’s based on the concept of a vocoder, which works by splitting a sound into different frequency bands, and then using the energy in those bands to modulate a sound like white noise or a sawtooth wave. I used the different flair values of the button as my frequency bands, i.e. the 60 s presses were mapped to a high-frequency band and the 0 s were mapped to a low frequency band. I counted the number of presses within each band within 3 hour windows and used these rates to modulate white noise. I also compressed the timescale so listening to the two-month dataset takes two minutes. I recommend playing the video below with headphones on, although you may want to turn the volume down at the beginning since it starts out loud.

The top plot is the sound waveform that you’re hearing, and the middle plot is like a spectrogram, showing the log-scaled rate of button presses over time. The bottom plots the instantaneous press rate for each flair over the 3 hour window used to compose each frame, so the minimum non-zero value is \log_{10}\left(\frac{1 \text{ press}}{180 \text{ minutes}}\right) \approx -2.3. Also, you can probably tell I cut out the first few days (60 hours to be exact), since the rate of button presses is so high at the beginning that it would’ve swamped out the rest of the audio. That and given you all hearing loss.

You can hear/see some pretty interesting things, like how the overall amplitude ebbs and flows over 24 hour periods, or how the flair values near the borders (e.g. 11 s, 21 s, etc.) stick out, almost like harmonics in a natural sound. You can even hear the hitchiker exodus, where a bunch of redditors pressed the button at 42 s. It ironically starts 42 s into the video (I swear that was unplanned).

Final Thoughts

I hope you enjoyed the examples and have a new-found appreciation for different ways of “displaying” your data. My examples here were all time-series data, but there’s no reason these techniques can’t be extended to other modalities, like spatial data from geographers or spectroscopy data from chemists. The sky’s the limit! (Get it? Because sound can’t propagate in outer space?)

I’ll see myself out.

Harnessing plasticity

Author’s note: This was an old journal entry from August 3, 2014.

An interesting article made it to the front page of the /r/techonology subreddit today about using vagal nerve stimulation (VNS) for the treatment of tinnitus. There seems to be a lot of redditors suffering from tinnitus, considering the top comment to for the practically every hearing-related submission to /r/science seems to be about tinnitus (see 1, 2, 3, and 4). I don’t know if this is from a preponderance of tinnitus sufferers on reddit, or whether redditors with tinnitus are just really vocal. Perhaps both.

Back to the more interesting issue. How crazy is it that VNS can potentially alleviate symptoms of tinnitus!? I read a bit about the proposed mechanism, namely modulating cortical plasticity via cholinergic projections from the nucleus basalis, which receive some input from the solitary nucleus. This certainly seems plausible, but it just seems crazy that stimulating the vagus nerve can have an effect at all on tuning in the auditory cortex, as shown in this paper. The treatment in that study involved pairing the VNS with tones, so it seems like there needs to some sort of interaction of bottom-up and top-down mechanisms to induce plastic changes. I find this interesting for two reasons:

  1. A method as crude as electrically stimulating the entire vagus nerve manages to successfully invoke the diffuse cholinergic projections to cortex, considering the vagus nerve performs so many different functions, and
  2. Activation of these very diffuse projections can have such a specific effect on one part of the brain.

Now this raises a lot of interesting questions. What other effects does VNS have? From the list of other uses on Wikipedia, potentially a lot. If the whole neocortex is made more plastic, are there system-wide changes going on, or does there need to be the bottom up input as well? Are the normal functions of the vagus nerve disrupted in any way? I think these types of questions are indicative of how much more we have to understand about the brain. You might argue (and I might agree) that perhaps it shouldn’t matter as long as it helps people and the negative side effects we observe are minimal. But that’s just it. How are we supposed to observe all (or at least an acceptable number) of the possible side effects if we have such a minimal understanding of the system?

There’s an interesting Latin maxim: ignoramus et ignorabimus (“we do not know and will not know”), credited to the 19th century German physiologist Emil du Bois-Reymond in Über die Grenzen des Naturerkennens (“On the limits of our understanding of nature”). This saying seems pretty defeatist, but sometimes I feel this applies to neuroscience, even if I don’t like especially like it. I tend to prefer David Hilbert’s retort: “Wir müssen wissen — wir werden wissen!” (“We must know — we will know!”). Of course, Hilbert also wanted to find a complete and consistent set of axioms for mathematics, which we now know is impossible (thanks Gödel!). Perhaps we could revise it to: “We really want to know, and we’ll probably know more in the future than now!” Not as catchy though.

Word Clouds!

Author’s note: This is an old journal entry from July 29, 2014 that I never did anything with, so now it’s going to become my first blog post.

I recently discovered Wordle, a site which allows you to create beautiful word clouds, a type of graphic consisting of words where the font size is proportional to how often the word appears in a given body of text. I thought it would be fun trying to use Wordle to analyze some abstract books from scientific conferences, but that may have been a bit overambitious. Who knew you could crash Chrome by trying to copy and paste 15,000+ pages of text?

Undeterred (i.e. too much time on my hands), I decided to write some Matlab code to make my own. This allowed me to analyze very large bodies of text and also have more control in the final graphic creation. I had to learn some regular expressions as well as use Matlab’s Computer Vision Toolbox in a manner in which it certainly wasn’t designed, but hey, it works. Below are two word clouds, showing the most common 200 words in the ARO 2014 and SfN 2013 abstract books, respectively. ARO stands for the Association of Research in Otolaryngology, and it’s definitely the largest conference devoted to hearing research. It pales in size, however, compared to the annual SfN meeting (Society for Neuroscience), which draws in about 30,000 people.

Word cloud from ARO 2014

Word cloud from ARO 2014

Word cloud from SfN 2013

Word cloud from SfN 2013

To make the word clouds, I simply counted word frequencies like Wordle, but in addition to ignoring common English words (“the”, “and”, etc.), I also removed nonspecific scientific words like “abstract” and “methods.” The result certainly isn’t as pretty as those from Wordle, but look, a brain! and sort of a cochlea! The clouds from Wordle have much nicer inter-word spacing due to the way they handle collision detection, but I had to be more flexible since I wanted the words to fit into an arbitrary shape.

Observations so far? Despite obtaining the #3 spot on the ARO cloud, the word “auditory” barely made it into the SfN cloud, achieving spot #196 and losing terribly to “visual” at #47. Oh well, at least the sensory neuroscientists can be bitter together, since they were beaten by other systems, namely #19 “memory” and #21 “motor.” In case you were wondering, the word “neurons” was used 17,452 times in the SfN abstract book, an order of magnitude increase from the #1 word in the ARO cloud: “cells,” which was used 1,846 times. This isn’t all that surprising given the huge size of SfN relative to ARO.

Lastly, for a summary of the data, here are some basic counts on the text analysis.