EyeWire Synesthesia mod

Chances are this will completely fail to satisfy the expectation but bare with me…
it is well known that the human brain does a great job at decomposing auditory information. We can follow someone on the phone despite traffic noise and recognize speech from noise radio receptions. This ability is only limited by the psychoacoustic masking effect and our individual hearing quality (ok… and it may depend a bit on one’s musicality ^^)

Some of you may already know what a spectrogram is (see http://en.wikipedia.org/wiki/Spectrogram). A spectrogram enables you to see what an audio intensity spectrum looks like at a given point of time and how it evolves in time (horizontal axis). It’s a 2D representation of a 1D dataset.
Synesthesia is a condition where sensory experiences mix, for example the number “5” is associated with a certain color or geometric shapes have a sound to them.

Let me propose that we dig a bit deeper into this experience by transforming the volume or 3d model information of neuronal structures into enhanced formats that we can experience in different ways. Literally listening to geometry may be an obvious approach to it. However I can imagine there are lots of different ways we can percieve whether a geometry “feels right”.
As a goal and motivation this may yield new techniques to enhance tracing speed / quality.

I’ll hear you around :wink:

Or maybe it would be better to provide an add-on interface for programmers to explore new visualization and UI concepts - that way you could tap into a whole new flavor of community skills.


This is a very interesting idea. I have heard of synesthesia and I do think mixing senses would be a cool thing to explore. Do you have any ideas on how this could be done?


@helge, for what it’s worth, we were looking into game design ideas in this vein, but it seems pretty hard to do. If you can figure out a way to make this work efficiently with good player accuracy, we’d assist you in accessing our API to build it (or if you just want to tool around, we’d help you out too). :slight_smile:

Will Silversmith
EyeWire Developer

This is an awesome idea! It would be most useful if perhaps the smoothness of a neuron’s surface could be measured in this way, as jagged areas in a neuron are most often areas of interest while tracing. Perhaps smoother surfaces could be associated with a more harmonious chord, and jagged areas a dissonant chord; or even smooth areas a low note and sharp surfaces higher notes. 

Representing the neuron as a whole this way would work very well as described - drawing attention to potentially problematic areas quickly - but it could also be effective if each slide was measured in this way, and scrolling up and down would change the chord/note accordingly. This would serve to draw attention accurately.

 This however, would be very misleading while learning the ropes of eyewire, and I would strongly advise that (if implemented) it not be used as a teaching tool. This is as it may teach players that jagged = bad, and not about mergers, how the AI interprets data and so on. For more experienced eyewirers, it would be very useful as for the reasons mentioned above.

These are just a few suggestions. I understand that this may be very difficult to implement, and I don’t want to put any more pressure on our wonderful devs at MIT :). This would add a whole new level to the game though, and is a very interesting idea to explore.

I’d love to hear more on what the rest of the community thinks.

Scythe & Moderator