This post is going to cover a very cool new project incorporating the myki light called the “Neurolyzer Table”. The table itself is a 2 foot cube with a polypropylene diffuser for the top. The table itself can be used as anything from a coffee table, to a stand for artwork such as glass or sculpture, to a backdrop for a retail store. But in this particular usage, the Neurolyzer is being used as a tool to explore the way that the visual cortex interprets lights strobing at very carefully chosen rates.

The result of looking directly into the Neurolyzer is a complex and incompletely understood interaction between the flickering stimulus, the eye, and the visual cortex. In summary, by strobing at particular frequencies the Neurolyzer sets up a standing wave inside of your brain that causes you to imagine seeing complex patterns. These are not identical to those seen on some psychedelic drugs, but may be similar in origin. Here is a video of the device in action at Boston Decom 2012.

The specific implementation shown here is based on a variety of other projects. In particular, the FnOREST project from Burning Man 2007 was an inspiration for the mechanism.

For a detailed neurological explanation of the behavior of this device, as well as technical details and code to develop your own system, please see below the break.

The following is split into two sections for clarity. An extensive background of the neurology behind the neurolyzer table in psychedelic mode is presented in the second section. It is super awesome, and a great read if you’re interested in the why of how the table works. There are some mathy parts, but also cool graphics! But to start, a story of how this particular project came to be. Example code is at the very end.

Note: There may be some discrepancies between the results reported in the first section and the theory in the background section. The difference between an experimentalist and a serious neuroscientist! As we said at the beginning – incompletely understood.

Build Story

(Build Story contributed by Brian Neltner)

First, credit where credit is due. The Neurolyzer Table in “psychedelic mode” uses strobing lights based loosely on the behavior of the FnOREST project, along with some fine tuning based on my personal preferences.

I first saw this effect at Burning Man in 2007 in the FnOREST project put on by Josh and Dustin. It was fantastic, and difficult to believe how intense it was. After several years of building LED lights, it occurred to me in the Fall that it might not require an entire foggy dome filled with ultrabright lights to get the same effect. So, I took one of the myki prototype lights and did some quick coding where it would slowly strobe through a range of strobe rates.

At first, I simply shone the light directly into my eyes with my eyes closed. The strobing effect through my eyelids indeed caused impressive effects, albeit not very colorful (mostly red).

Then I took a piece of ABS plastic I had laying around and put it above the light as a diffuser. With the diffuser to decrease the intensity of the light I was able to look at the light with my eyes open, which made a much more colorful scene! I reprogrammed the light to alternate between either opposite hues, or “primary colors” (red and green) to behave similarly to the Fnorest system. I kept the strobe rate shifting in place. With this system, despite a small piece of plastic only 1 foot on a side, I was able to get very strong geometric effects to appear. From estimating the effective strobe rates, it appeared that strobe rates between 60Hz and 100Hz were the most effective for causing a dramatic effect.

The next step was to try to build a more robust mounting mechanism for the plastic diffuser piece to make it portable. At present I was just holding a piece of plastic above the light, so the next logical step was to take a simple and inexpensive 2 foot cube cardboard box and convert it into a chassis. One myki light is installed in the center of the bottom, and a polypropylene sheet is used on the top to act as a uniform field for displaying colors.

Several other types of diffusers were used, from etched glass to polyethylene to ABS. However, most other plastics were either too transparent, too opaque, or else had structure which scattered light in a way that was distracting. I found experimentally that a diffuser that had patterns in it such as paper made it so that my eyes would focus on the objects on the diffuser and make the effect much more difficult to see.

I took one of the prototype myki systems with a flange and used tape to affix it fairly permanently to the bottom of the box, and cut a small hole in the side for running the power supply cable (with strain relief). I programmed it up with some experimental code that strobed between two hues 120 degrees apart that slowly rotated around the color wheel while also varying the overall strobe rate. Then it was action time.

With the 22 degree half angle lens installed in the prototype, the 2 foot plastic piece was fairly uniformly illuminated, and far more brightly than I would have expected from a single light. I installed the initial test code on the box, and let’s just say… the result was impressive! The box was demonstrated at Brickbottom Open Studios in November 2012, to the great enjoyment of everyone. One of my goals with my artwork is to produce things which can keep a nine year old’s attention. I feel like if it can do that, then it’s guaranteed to be interesting to everyone (whether they admit it or not). And let’s just say a lot of kids had a lot of fun.

I did notice in this period that there were segments of the strobe rate that did not seem to cause an effect for anyone. At the low end, eventually people just saw it as two discrete colors being displayed and didn’t see any patterns. At the high end, people eventually stopped being able to tell that it was strobing and just saw a slightly flickering pastel. Great for a first experiment!

For the next experiment, I modified the code to narrow the strobe sweep to a narrower band, approximately 20Hz to 50Hz. Under these conditions, I found that essentially anyone looking at it at any point in the sweep would be affected. From there, I took the device to Boston Decom 2012, where it seemed to attract a great deal of amusement.

I’ve also found that most people who look at the system report seeing colors which are not there. It’s quite difficult to describe, but what it looks like to me is that it will, for instance, be strobing between cyan and magenta. And then suddenly I see a strange blue or red. It’s such a short flicker that I’m confident it’s not actually in the spectrum at all but is instead some sort of artifact similar to impossible colors, imaginary colors, or another perceptual artifact.

Neurological Background

(Neurological Background contributed by Michael Rule)

Geometric hallucinations induced by flickering light

Most people see colorful geometric patterns when looking at uniform flickering lights of a certain frequency. These are sometimes called “flicker phosphene hallucinations”. These patterns are subjectively described as “checkerboards” “ripples” “circles” “starbursts” “honeycombs”, taking on an overall tunnel-like appearance. It is striking how universal these patterns are – these simple flicker-induced geometric hallucinations are similar across individuals, and are similar to some visual hallucinations seen in visual-triggered epilepsy, migraine auras, and hallucinogens. Indeed, flicker phosphenes are presumed to be the same as the flicker-triggered seizures, but only progress to out-of-control activity in individuals with epilepsy. A simple way to achieve such hallucinations is to close one’s eyes and face toward the sun. Then, with fingers outstretched and separated, wave the hand back and forth in front of the eyes. Your fingers will create a flickering of the sunlight, which will be diffused and made uniform by closed eyelids.

The shortest primer on the brain and visual system ever

To understand how flickering lights can trigger geometric hallucinations, and why the hallucinations take on particular forms, we need to understand a little about the nervous system. I attempt to give a short overview here.

Very briefly, the eye and the brain contain cells called neurons which are specialized for computation. Neurons communicate with each other in all-or-nothing events called action potentials or “spikes”, which propagate along “wires” called axons, extending away from the cell body. These spikes are picked up by other neurons with more local tree of wires called dendrites. Early visual processing relates to the sort of geometric hallucinations that we can get out of the early visual system. Light enters the eyes and is picked up by photoreceptor cells in the retina. These cells communicate to neurons in the retina that compress visual input, which is sent to visual cortex for further processing. We will focus on four aspects of this compression related to flicker induced geometric visual hallucinations:

1. The RGB color-space picked up by the photo-receptors is converted to three contrasts: light-dark, yellow-blue, and red-green. The color-space conversion relates to how, for instance, a red-green flicker is especially effective at eliciting hallucinations.
2. The image is converted to a representation that focuses on edges and contrasts – uniform areas are ignored. This band-pass present in early visual processing relates to how neurons are connected in the retina and visual cortex, which in turns determines what size “waves” might appear in geometric visual hallucinations.
3. The retina has band-pass properties in time as well: slowly changing visual inputs are not registered. This is important for the temporal aspect of flicker induced hallucinations. Certain speeds of flickering will be picked up and transmitted better.
4. The eye has the greatest resolution in the very center, and the resolution drops off toward the periphery. This is why, for example, it is difficult to read things out of the corner of your eye. The brain also devotes less space to processing the periphery. The way that the visual field is “unfolded” in the brain determines the subjective appearance of spontaneous (hallucinated) neural activity.

A theory of geometric visual hallucinations

A theory on the origin of geometric visual hallucinations comes from Ermentrout and Cowen (1979), and briefly, makes the following hypotheses:

1. The tunnel-like appearance of the overall hallucination relates to how the brain maps the visual field.
2. Patterns like checkerboards and honeycombs arise from superposition of multiple waves
3. Local interactions in the brain generate wave-like patterns of activity

The two halves of the visual field are split apart and each processed in one half of the brain. The brain devotes more processing power toward the center of the visual field. The net effect is to “unfold” the circular visual field from the retina into a “strip” of the brain at the back called the primary visual cortex. The result is that tunnel-like visual inputs map to simple regular patterns in the brain, and the inverse: hallucinated simple regular patterns in visual cortex are subjectively experienced as tunnel-like visual inputs.

Single plane waves of activity in the brain can explain circular, spiral, and starburst hallucinations. Checkerboards can be explained as the sum of two waves at right-angles, and honeycombs can be explained as the sum of three evenly spaced plane waves.

The conditions under which the brain spontaneously forms waves of activity are an open question. Qualitatively, one can think of it as the brain trying to “find” patterns in an image based on the development of the eye over time. Under certain circumstances, the pattern finding circuitry becomes unstable, and tends toward these simple, wave-like patterns.

Driving the brain with a uniform, flickering input

Neural activity in the brain oscillates spontaneously. There is a natural 8-12Hz oscillation in the visual system called the alpha rhythm. Providing visual input at alpha frequency excites this oscillation. Stimulation at 8-25Hz also works to excite the alpha oscillation. Even though this is twice the intrinsic frequency of the system, local patches of the brain are hypothesized to respond only to every-other oscillation cycle, effectively oscillating at alpha. The percise optimal frequencies vary from person to person, and there may be other processes contributing to flicker hallucinations that we have yet to understand. Interestingly, different frequencies favor different patterns.

Why does making visual cortex “resonate” lead to pattern formation? The qualitative idea is quite simple: consider shaking a tray of water up and down. Even though you are forcing the whole tray uniformly, if you hit the right frequencies, you can create standing waves. The visual cortex does something similar when you hit it with flickering light. When you average out the details of the brain, it turns out it’s a lot like any other excitable, oscillatory, medium. Flicker phosphenes can be thought of like standing waves arising in a periodically forced fluid – water in a speaker cone, for example. My personal favorite, however, is the nonlinear: cornstarch in water

Mathematical Model

It is possible to build a mathematical model that, to a first approximation, captures the response of visual cortex to flickering stimuli. This model is taken from this paper. Modeling individual neurons is very difficult, and accurate modeling a network the size of the visual cortex is not yet possible. Instead, we will look at the very fuzzy average behavior of neurons. This will be sufficient to capture wave dynamics that are large compared to the size of indivisual cells. This is approach is a type of “mean field” approximation. We use very basic assumptions:

1. There are two populations of cells that matter: inhibitory and excitatory (call them $E$ and $I$)
2. At each point in the visual cortex, a neural population can be summarized by an “average firing rate”. Represent these rates as functions over two dimensions $E(x,y)$ and $I(x,y)$.
3. Cells connect to each-other locally, with inhibition spreading further than excitation. Approximate these connections with Gaussian convolutions.
4. A neural population sums its inputs linearly
5. Neural populations apply a soft thresholding nonlinearity to map from input to firing rate
6. Average neural firing rates evolve smoothly, with excitation a bit faster than inhibition

These assumptions are enough to begin building a model. Cells connect to each other locally, modeling connections as Gaussian: $$W_e = K_e(x,y) \ast E(x,y); K_e(x,y) = \frac{1}{\sigma_e \sqrt{2\pi}}exp \left(-\frac{x^2+y^2}{2 \sigma_e^2} \right)$$ $$W_i = K_i(x,y) \ast I(x,y); K_i(x,y) = \frac{1}{\sigma_i \sqrt{2\pi}}exp \left(-\frac{x^2+y^2}{2 \sigma_i^2} \right)$$ A neural population sums its inputs linearly, with some unknown weighting $$E_{input} = a_{ee} W_e - a_{ie} W_i$$ $$I_{input} = a_{ei} W_e - a_{ii} W_i$$ Neural populations apply a soft thresholding nonlinearity to map from input to firing rate. Let’s use $f(x)=1/(1+exp(-x))$ for a nonlinearity, and give it an adjustable threshold value $t$: $$E_{activation} = f(E_{input} - t_e)$$ $$I_{activation} = f(I_{input} - t_i)$$ Add in stimulus input terms $$E_{activation} = f(E_{input} - t_e + s_e (t) )$$ $$I_{activation} = f(I_{input} - t_i + s_i (t) )$$ Average neural firing rates evolve smoothly $$\tau_e \dot E = -E + a_e E_{activation}$$ $$\tau_i \dot I = -I + a_i I_{activation}$$ This is the model. Expanding it $$\tau_e \dot E = -E + f( a_{ee} W_e - a_{ie} W_i - t_e + s_e (t) )$$ $$\tau_i \dot I = -I + f( a_{ei} W_e - a_{ii} W_i - t_i + s_i (t) )$$ To see whether this system “hallucinates”, one can look for the emergence of waves in the solution to these equations over time. The simplest way to get started is to simulate these equations and play with the parameters and see what happens. You should find that there are parameter regimes where waves form spontaneously – and others where waves can only be supported by periodic forcing. These would correspond to “spontaneous” and “flicker-induced” geometric hallucinations, respectively. This paper gives some starting parameters of $\tau_e=10$ $\tau_i=20$ $a_{ee}=10$, $a_{ei}=12$, $a_{ie}=8.5$, $a_{ii}=3$, $t_e=3.5$, $t_i=3.5$, $\sigma_i = 2.5 \sigma_e$. We found that this system prefers to form stripes and honeycombs – no checkerboards. Maybe you can find a similar system that supports checkerboards as well?

Understanding the behavior of this system is a challenge since there is a time-varying periodic input. This means that one cannot examine whether spatial waves form from the rest state (there is no rest state). We need to convert our system to a representation that incoroporates the time varying input. Since the input is periodic, one can use floquet theory to convert these equations into a linear map. This map represents the evolution of the system over one period of forcing. In particular, we will start with the spatially uniform state, and ask whether this state is stable over time. One way to phrase this is to ask if there is any “mixing” between the spatially uniform mode and “wave” modes over the course of one oscillation – and whether this mixing will cause the wave modes to grow over time. Another way to think of this is that the periodic stimulus briefly pushes the brain through a state where it is spontaneously pattern forming, and the map summarizes the net effect of this over one cycle of stimulation. When we do this, we find that under certain frequencies of periodic forcing, a single spatial frequency begins to grow – these are the conditions needed to explain a flicker-induced hallucination. More details of this treatment, which uses a mixture of algebraic and numerical techniques, can be found in this article.

Neurological Background Summary

Some flicker-induced geometric visual hallucinations can be thought of as “ripples” and “standing waves” excited by the flickering light, occurring in the brain. The model outlined here captures wave modes, but there are many open questions. For example, no mathematical model has treated the interaction of flicker phosphene hallucinations with color perception, or interactions with finer-scale neural connectivity in visual cortex related to extraction of intermediate visual features. The phenomena is an interesting “hack” that reveals the inner workings of the early human visual system without the need for “pharmacological” assistance, and may be a useful tool in psychophysics for noninvasively probing early vision.

Finally, the source code!

Well, enough theorizing. Try it out and see if you can make it work better!

You can find below the arduino leonardo compatible code which I used on the myki light in order to produce a good effect. It can definitely still be improved – for instance, unsurprisingly some hue combinations (red and green in particular) are much more intense than others. This seems explainable from the neurology background above. So, there is probably a better way to pick hues.

Disclaimer: DO NOT TRY THIS IF YOU ARE EPILEPTIC! THESE FREQUENCIES ARE EVEN WORSE THAN MOST STROBING LIGHTS!

*********************************
* Created 23 August 2012        *
* Copyright 2012, Brian Neltner *
* http://saikoled.com           *
* Licensed under GPL3           *
*********************************

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

A copy of the GNU General Public License can be found at
http://www.gnu.org/licenses/gpl.html

Description:
This sketch implements psychadelic mode, inspired by Fnorest aka Seizure Dome.
It has some legacy code that's not in use that was in my default template for 
working with the saikoduino.

*/

#include "math.h"
#define DEG_TO_RAD(X) (M_PI*(X)/180)

#define redPin 9    // Red LED connected to digital pin 9
#define greenPin 10 // Green LED connected to digital pin 10
#define bluePin 11 // Blue LED connected to digital pin 11
#define whitePin 13 // White LED connected to digital pin 13
long int modifier; // Strobe shift amount.
long int delaytime;  // Time between strobe shifts changes.

int color1;
int color2;

struct HSI {
  float h;
  float s;
  float i;
  float htarget;
  float starget;
} color;

void sendcolor() {
  int rgbw[4];
  while (color.h >=360) color.h = color.h - 360;
  while (color.h < 0) color.h = color.h + 360;
  if (color.i > 1) color.i = 1;
  if (color.i < 0) color.i = 0;
  if (color.s > 1) color.s = 1;
  if (color.s < 0) color.s = 0;
  // Fix ranges (somewhat redundantly).
  hsi2rgbw(color.h, color.s, color.i, rgbw);
  analogWrite(redPin, rgbw[0]);
  analogWrite(greenPin, rgbw[1]);
  analogWrite(bluePin, rgbw[2]);
  analogWrite(whitePin, rgbw[3]);
}

void hsi2rgbw(float H, float S, float I, int* rgbw) {
  int r, g, b, w;
  float cos_h, cos_1047_h;
  H = fmod(H,360); // cycle H around to 0-360 degrees
  H = 3.14159*H/(float)180; // Convert to radians.
  S = S>0?(S<1?S:1):0; // clamp S and I to interval [0,1]
  I = I>0?(I<1?I:1):0;
  
  // This section is modified by the addition of white so that it assumes 
  // fully saturated colors, and then scales with white to lower saturation.
  //
  // Next, scale appropriately the pure color by mixing with the white channel.
  // Saturation is defined as "the ratio of colorfulness to brightness" so we will
  // do this by a simple ratio wherein the color values are scaled down by (1-S)
  // while the white LED is placed at S.
  
  // This will maintain constant brightness because in HSI, R+B+G = I. Thus, 
  // S*(R+B+G) = S*I. If we add to this (1-S)*I, where I is the total intensity,
  // the sum intensity stays constant while the ratio of colorfulness to brightness
  // goes down by S linearly relative to total Intensity, which is constant.

  if(H < 2.09439) {
    cos_h = cos(H);
    cos_1047_h = cos(1.047196667-H);
    r = S*255*I/3*(1+cos_h/cos_1047_h);
    g = S*255*I/3*(1+(1-cos_h/cos_1047_h));
    b = 0;
    w = 255*(1-S)*I;
  } else if(H < 4.188787) {
    H = H - 2.09439;
    cos_h = cos(H);
    cos_1047_h = cos(1.047196667-H);
    g = S*255*I/3*(1+cos_h/cos_1047_h);
    b = S*255*I/3*(1+(1-cos_h/cos_1047_h));
    r = 0;
    w = 255*(1-S)*I;
  } else {
    H = H - 4.188787;
    cos_h = cos(H);
    cos_1047_h = cos(1.047196667-H);
    b = S*255*I/3*(1+cos_h/cos_1047_h);
    r = S*255*I/3*(1+(1-cos_h/cos_1047_h));
    g = 0;
    w = 255*(1-S)*I;
  }
  
  rgbw[0]=r;
  rgbw[1]=g;
  rgbw[2]=b;
  rgbw[3]=w;
}

void setup() {
  // Configure LED pins.
  pinMode(whitePin, OUTPUT);
  pinMode(redPin, OUTPUT);
  pinMode(greenPin, OUTPUT);
  pinMode(bluePin, OUTPUT);
  
  digitalWrite(whitePin, 0);
  digitalWrite(redPin, 0);
  digitalWrite(greenPin, 0);
  digitalWrite(bluePin, 0);
  
  // Set prescalers to 1 for all timers; note that this means that millis() et all will be 64x too short.
  TCCR0B = _BV(CS00);
  TCCR1B = _BV(CS10);
  TCCR3B = _BV(CS30);
  TCCR4B = _BV(CS40);
  
  color.i = 1;
  color.s = 1;
  
  color1 = 0;
  color2 = 120;
  
  delaytime = 900;
  modifier = 10;
}

void loop() {
  color.h = color1;
  sendcolor();
  delay(delaytime);
  color.h = color2;
  sendcolor();
  delay(delaytime);
  delaytime = delaytime + modifier;
  if (delaytime > 2000 or delaytime < 900) modifier = -modifier;
  color1 += 1;
  color2 += 1;
  while (color1 >= 360) color1 = color1 - 360;
  while (color2 >= 360) color2 = color2 - 360;
}