3 of 3: Brain Hacking

The hospital doesn’t have the equipment to decrypt and download the actual data. But Jane knows that the LoTeks can, so they drive to the ruined bridge that is the LoTek home base. As mentioned earlier under Door Bombs and Safety Catches the bridge guards nearly kill them due to a poorly designed defensive system. Once again Johnny is not impressed by the people who are supposed to help him.

When Johnny has calmed down, he is introduced to Jones, the LoTek codebreaker who decrypts corporate video broadcasts. Jones is a cyborg dolphin. Continue reading

Brain Scanning

The second half of the film is all about retrieving the data from Johnny’s implant without the full set of access codes. Johnny needs to get the data downloaded soon or he will die from the “synaptic seepage” caused by squeezing 320G of data into a system with 160G capacity. The bad guys would prefer to remove his head and cryogenically freeze it, allowing them to take their time over retrieval.

1 of 3: Spider’s Scanners

The implant cable interface won’t allow access to the data without the codes. To bypass this protection requires three increasingly complicated brain scanners, two of them medical systems and the final a LoTek hacking device. Although the implant stores data, not human memories, all of these brain scanners work in the same way as the Non-invasive, “Reading from the brain” interfaces described in Chapter 7 of Make It So.

The first system is owned by Spider, a Newark body modification
specialist. Johnny sits in a chair, with an open metal framework
surrounding his head. There’s a bright strobing light, switching on
and off several times a second.

jm-20-spider-scan-a

Nearby a monitor shows a large rotating image of his head and skull, and three smaller images on the left labelled as Scans 1 to 3. Continue reading

The Memory Doubler

In Beijing, Johnny steps into a hotel lift and pulls a small package out his pocket. He unwraps it to reveal the “Pemex MemDoubler”.

jm-4-memdoubler-a

Johnny extends the cable from the device and plugs it into the implant in his head. The socket glows red once the connection is made.

jm-4-memdoubler-b-adjusted

Continue reading

Itchy’s SFW Masturbation Chair

With the salacious introduction, “Itchy, I know what you’d like,” Saun Dann reveals himself as a peddler of not just booby trapped curling irons, but also softcore erotica! The Life Day gift he gives to the old Wookie is a sexy music video for his immersive media chair.

SWHS-Chair-03

The chair sits in the family living room, and has a sort of helmet fixed in place such that Itchy can sit down and rest his head within it. On the outside of the helmet are lights that continuously blink out of sync with each other and seem unrelated to the actual function of the chair. Maybe a fairy-lights power indicator?

SWHS-Chair-02

Saun first powers the device by inserting a “proton pack” into the back of the chair. This is kind of strange since none of the other devices seen in the home require batteries or charging. Are they lower power so batteries last longer? Is there an unseen electrical infrastructure but that monitors any plugged in object for illegal-device signatures? Whatever the reason, the battery pack plugs in and the chair comes to life.

When Itchy sits down, he rests his head in the helmet, and Saun puts a media cartridge into a tray sticking out of the armrest. There is a single red button on the forward edge. He then engages the cartridge by slapping it on edge so the tray slides into a recess. Then he lowers the forehead plate of the helmet over Itchy, who sits back to enjoy the show. Saun wishes him, “Happy Life Day!” and then with a nudge-nudge-wink-wink-you-know-seeeeexxxxx tone in his voice, leans in to reiterate, “And I do mean Happy Life Day!”

SWHS-Chair-09

Itchy punches the top of the armrest with his hairy finger, and the media plays, full of theremin meanderings, stage lights and spandex dancers shot through a kaleidoscope-refraction lens, and a scantily-feathered singer (IMDB tells me the character name is Mermeia Holographic Wow, played by the diva Diahann Carroll) who purrs out a full 5 minutes of mind-numbing introduction before a 4-minute musical number. You can almost hear the director saying over a soggily-chewed stogie, “Sorry, Diahann, but we don’t have a lot to work with, here, you’re really going to have to stretch this out.”

I know you’re searching for me
Searching
Searching
I am here
My voice is for you alone
I am found in your eyes only
I exist for you
I am in your mind
As you create me
Oh yes
I can feel my creation
[Giggle]
I’m getting your message
Are you getting mine?

Itchy growls and spasms in his chair enthusiastically.

Oh, oh! We are excited aren’t we?
Well just relax
Just relax
Yes

[More Itchy grunting]

Now
We can have a good time
Can’t we?

[spasming, panting]

I’ll tell you a secret
I find you adorable

Itchy loves this assertion so much that he punches a control on the armrest, and the playback jumps back to replay the line again. And again. And again. Four times in total.

(I find you adorable)
(I find you adorable)
(I find you adorable)
I don’t need to ask how you found me
You see, I am your fantasy
I am your experience
So experience me
I am your pleasure
So enjoy me
This is our moment together in time
That we might turn this moment an eternity

[Music mercifully begins]

SWHS-Chair-07

I’m sorry to have to remind people about this point in your career, Diahann.

So…Exactly what is this machine?

So, if the words of the recording are to be believed, this machine automatically reads the mind (or stored preferences) of its user to create a custom, on-the-fly immersive video.

Here you should note that Itchy’s species of sexual preference is human, not Wookie, and then note with sadness that Chewie is not mixed-species. Itchy is living an unfulfilled life.

If it constructs its visions on-the-fly, then what is the cartridge for? It can’t be Itchy’s preferences. Why would Saun have them? My best guess is that the cartridge contains the template of the song, This Minute Now, and the device reads the wearer’s in-the-moment preferences to pick the avatar that sings it. That template isn’t neutral, though. It has to be a sexy template, because Saun does all the nod’s as good as a wink to a blind bat routine. Aww, yeah, sexy template for Life Day.

Knowing that this might be a privately-observed sexytime moment, I’d recommend the designers add a curtain or perhaps situate the chair inside a private chamber to make the user less exposed. Otherwise it might be awkward—for a human user at least—for other members of the family to observe him becoming visibly and audibly aroused (the panting), much less do…uh…anything about it. Of course I don’t know Wookie social rules, so this might be well within their social norms. Kinda makes you wonder about how Chewie spends his idle time on the Falcon in front of Han. “Chewie. Do you mind? Take it into your berth!”

Controls

The instant replay feature is useful to the task. And it’s quite well executed, since it’s a quick-to-press button to replay the last moment, and that moment is of unspecified length. The media must have very sophisticated and detailed markup for a “repeat that bit” to work, and it does. Of course, it also can read Itchy’s mind, so maybe it just knows to play across the most-recent high-excitement part. It’s a self-administered dopamine hit.

A better tool might monitor the user’s brainwaves for a perfect combination of tension and release to ensure a perfectly satisfying experience. No button needed. That would also alleviate the problem that the user’s hand might be otherwise engaged during the exciting part to try and target a button. The (prolly NSFW text, even if it’s WebMDsexual response cycle of humans is a known thing, so surely a Wookie’s is too. Let me disturb you by visualizing the combination of concepts so implied.

Wookie-Response-Cycle

But we also have to ask after the placement and purpose of that sole red button. Saun slapped the cartridge edge-on to lodge the tray in place. But that’s where the button is. Did the lodging activate the play, or did pressing the button?

If the cartridge, what does the button do? (Not replay, Itchy clearly presses the top of the armrest for that.) And how does it avoid accidental activation while being slapped in?

If the button, why have the tray slide in and out? To keep it private? Sorry, you lost that battle with the living room masturbatorium. To protect the media? Then why have it in the armrest where it’s sure to be subject to the bangings of Wookie demands for again!?

With the commercial release of Oculus Rift just about to ship post-CES, let’s not turn to this device for any immersive-media lessons. There is better blind-use masturbation VR in Strange Days, more dystopic ones in THX-1138 (yes, I now realize, it’s a recurring Lucas theme), more private ones in Sleeper, and less creepy things almost anywhere you turn.

Let’s just let Itchy have his personalized-avatar, happy Life Day, there, in the middle of the astroturfed family room.

SWHS-Chair-10

Stay classy, Star Wars Holiday Special.

Dat glaive: Projectile gestures

TRIGGER WARNING: IF YOU ARE PRONE TO SEIZURES, this is not the post for you. In fact, you can just read the text and be quit of it. The more neurologically daring of you can press “MORE,” but you have been forewarned.

If the first use of Loki’s glaive is as a melée weapon, the second use is of a projectile weapon. Loki primes it, it glows fiercely blue-white, and then he fires it with usually-deadly accuracy to the sorrow of his foes.

This blog is not interested in the details of the projectile, but what is interesting is the interface by which he primes and fires it. How does he do it? Let’s look. He fires the thing 8 times over the course of the movie. What do we see there? Continue reading

Brain interfaces as wearables

There are lots of brain devices, and the book has a whole chapter dedicated to them. Most of these brain devices are passive, merely needing to be near the brain to have whatever effect they are meant to have (the chapter discusses in turn: reading from the brain, writing to the brain, telexperience, telepresence, manifesting thought, virtual sex, piloting a spaceship, and playing an addictive game. It’s a good chapter that never got that much love. Check it out.)

This is a composite SketchUp rendering of the shapes of all wearable brain control devices in the survey.

This is a composite rendering of the shapes of most of the wearable brain control devices in the survey. Who can name the “tophat”?

Since the vast majority of these devices are activated by, well, you know, invisible brain waves, the most that can be pulled from them are sartorial– and social-ness of their industrial design. But there are two with genuine state-change interactions of note for interaction designers.

Star Trek: The Next Generation

The eponymous Game of S05E06 is delivered through a wearable headset. It is a thin band that arcs over the head from ear to ear, with two extensions out in front of the face that project visuals into the wearer’s eyes.

STTNG The Game-02

The only physical interaction with the device is activation, which is accomplished by depressing a momentary button located at the top of one of the temples. It’s a nice placement since the temple affords placing a thumb beneath it to provide a brace against which a forefinger can push the button. And even if you didn’t want to brace with the thumb, the friction of the arc across the head provides enough resistance on its own to keep the thing in place against the pressure. Simple, but notable. Contrast this with the buttons on the wearable control panels that are sometimes quite awkward to press into skin.

Minority Report (2002)

The second is the Halo coercion device from Minority Report. This is barely worth mentioning, since the interaction is by the PreCrime cop, and it is only to extend it from a compact shape to one suitable for placing on a PreCriminal’s head. Push the button and pop! it opens. While it’s actually being worn there is no interacting with it…or much of anything, really.

MinRep-313

MinRep-314

Head: Y U No house interactions?

There is a solid physiological reason why the head isn’t a common place for interactions, and that’s that raising the hands above the heart requires a small bit of cardiac effort, and wouldn’t be suitable for frequent interactions simply because over time it would add up to work. Google Glass faced similar challenges, and my guess is that’s why it uses a blended interface of voice, head gestures, and a few manual gestures. Relying on purely manual interactions would violate the wearable principle of apposite I/O.

At least as far as sci-fi is telling us, the head is not often a fitting place for manual interactions.

The secret of the tera-keyboard

GitS-Hands-01

Many characters in Ghost in the Shell have a particular cybernetic augmentation that lets them use specially-designed keyboards for input.

Triple-hands

To control this input device, the user’s hands are replaced with cybernetic ones. Normally they look and behave like normal human hands. But when needed, the fingers of these each split into three separate mini-fingers, which can move independently. These 30 spidery fingerlets triple the number of digits at play, dancing across the keyboard at a blinding 24 positions per second.

GitS-Hands-02

The tera-keyboard

The keyboards for which these hands were built have eight rows. The five rows nearest the user have single symbols. (QWERTY English?) Three rows farthest from the user have keys labeled with individual words. Six other keys at the top right are unlabeled. Each key glows cyan when pressed and is flush with the board itself. In this sense it works more like a touch panel than a keyboard. The board has around 100 keys in total.

GitS-Hands-03

What’s nifty about the keyboard itself is not the number of keys. Modern keyboards have about that many. What’s nifty is that you can see these keyboards are massively chorded, with screen captures from the film showing nine keys being pressed at once.

GitS-Hands-04

Let’s compare. (And here I owe a great mathematical debt of thanks to Nate Clinton for his mastery of combinatorics.) The keyboard I’m typing this blog post on has 104 keys, and can handle five keys being pressed at once, i.e, a base key like “S” and up to four modifier keys: shift, control, option, and command. If you do the math, this allows for 1600 different keypresses. That’s quite a large range of momentary inputs.

But on the tera-keyboard you’re able to press nine keys at once, and more importantly, it looks like any key can be chorded with any other key. If we’re conservative in the interpretation and presume that 9 keys must be pressed at once—leaving 6 fingerlets free to move into position for the next bit of input—that still adds up to a possible 2,747,472,247,520 possible keypresses (≈2.7 trillion). That’s about nine orders of magnitude more than our measley 1600. At 24 keypresses per second, that’s a data rate of 6.5939334e+13 per second.

GitS-Hands-05

So, ok, yes, fast, but it only raises the question:

What exactly is being input?

It’s certainly more than just characters. Unicode‘s 110,000 characters is a fraction of a fraction of this amount of data, and it covers most of the world’s scripts.

Is it words? Steven Pinker in his book The Language Instinct cites sources estimating the number of words in an educated person’s vocabulary is around 60,000. This excludes proper names, numbers, foreign words, any scientific terms, and acronyms, so it’s pretty conservative. Even if we double it, we’re still around the number of characters in Unicode. So even if the keyboard had one keypress for every word the user could possibly know and be thinking at any particular moment, the typist would only be using a fragment of its capacity.

typing

The only thing that nears this level of data on a human scale is the human brain. With a common estimate of 100 billion neurons, the keyboard could be expressing the state of it’s users brain, 24 times a second, distinguishing between 10 different states of each neuron.

This also bypasses one of the concerns of introducing an input mechanism like this that requires active manipulation: The human brain doesn’t have the mechanisms to manage 30 digits and 9-key-chording at this rate. To get it to where it could manage this kind of task would need fairly massive rewiring of the brain of the user. (And if you could do that, why bother with the computer?)

But if it’s a passive device, simply taking “pictures” of the brain and sharing those pictures with the computer, it doesn’t require that the human be reengineered, just re-equipped. It requires a very smart computer system able to cope with and respond to that kind of input, but we see that exact kind of artificial intelligence elsewhere in the film.

The “secret”

Because of the form factor of hands and keyboard, it looks like a manual input device. But looking at the data throughput, the evidence suggests that it’s actually a brain interface, meant to keep the computer up to date with whatever the user is thinking at that exact moment and responding appropriately. For all the futurism seen in this film, this is perhaps the most futuristic, and perhaps the most surprising.

GitS-Hands-06

Thermoptic camouflage

GitS-thermoptic-03

Kusanagi is able to mentally activate a feature of her skintight bodysuit and hair(?!) that renders her mostly invisible. It does not seem to affect her face by default. After her suit has activated, she waves her hand over her face to hide it. We do not see how she activates or deactivates the suit in the first place. She seems to be able to do so at will. Since this is not based on any existing human biological capacity, a manual control mechanism would need some biological or cultural referent. The gesture she uses—covering her face with open-fingered hands—makes the most sense, since even with a hand it means, “I can see you but you can’t see me.”

In the film we see Ghost Hacker using the same technology embedded in a hooded coat he wears. He activates it by pulling the hood over his head. This gesture makes a great deal of physical sense, similar to the face-hiding gesture. Donning a hood would hide your most salient physical identifier, your face, so having it activate the camouflage is a simple synechdochic extension.

GitS-thermoptics-30

The spider tank also features this same technology on its surface, where we learn it is a delicate surface. It is disabled from a rain of glass falling on it.

GitS-spidertank-01

This tech less than perfect, distorting the background behind it, and occasionally flashing with vigorous physical activity. And of course it cannot hide the effects that the wearer is creating in the environment, as we see with splashes the water and citizens in a crowd being bumped aside.

Since this imperfection runs counter to the wearer’s goal, I’d design a silent, perhaps haptic feedback, to let the wearer know when they’re moving too fast for the suit’s processors to keep up, as a reinforcement to whatever visual effects they themselves are seeing.

UPDATE: When this was originally posted, I used the incorrect concept “metonym” to describe these gestures. The correct term is “synechdoche” and the post has been updated to reflect that.