Tactile USER Interface (TUI) DEVELOPMENT
A good technology idea that has not been done in any highly recognized way so far would be a platform that allows you to develop completely original custom software from scratch that has the ability to communicate completely tactile-based interfaces through devices or controllers.
Similar Current Technologies
In order to do this with technology we already have would be to combine music controllers with audio feedback, with keyboard keys that are very easy to learn without vision, and the audio pings and unique sounds that identify the actions going on on the device or computer, so that one is able to employ the same amount of power as a graphics user interface (GUI) but instead through a TUI. The challenge here is having the same level of flexibility in programming applications that can all use the same tactile feedback system while simultaneously controlling the device or computer with the same area of sensory, being tactile which is usually used for control of devices that interact with the interface rather than direct tactile control over the software, similar to how a music or synthesizer controller can match software keys and knobs to give a tactile interface to the software instruments. Except that instead of it being targeted to one medium such as music applications, but having that kind of control for multiple applications that are perfectly fit to them, through innovative science in conjunction with haptic feedback and robotics.
Based on my research, there was a particular boom in advanced tactile, tangible and haptics feedback and control in 2012 with many promising technologies being developed, prototyped and presented at that time. Since then the field has been quiet with more papers done by University and college students being done on the subject than actual implementations being used in the market or taken on by scientists, engineers and developers. The iPad was rumoured to have more advanced haptics such as found in this Forbes article, while out-of-the-box “organic” tangible controllers and interfaces have also been under development by MIT’s Tangible Media Group. Unfortunately actual practical daily use of such tools with multiple new applications have not been as easily recognized or promoted as these development are still hit-or-miss, young and raw.
Potential Solution: Reconfigurable Brail
I imagine a robotic device at heart where brail or equivalent code could be generated or reconfigured. In order to overcome the issue of control against interface feedback, since design is complicated by needing both to be handled in the tactile sensory domain, is instead to switch the sensory domain for control from tactile to one or a combination of other sensory mechanisms. The first I thought of was voice control as we have for the famous Siri or the original speech recognition software Dragon Speech for writing with your voice. Ideally you could “type” brail with your voice and read it back immediately with the robotic reconfigurations of the brail on the device. These interfaces, to remain simpler at first while still being smart would be to go further with one specific aspect of recent technology found in the gaming industry.
Pushing the Latest Gaming Innovations
The Nintendo Switch’s new default controller setup, the joy-cons include advanced “HD rumble” technology which allows for a higher fidelity of rumble feedback experiences to enhance the entertainment and augment the interactivity of their games. By taking this a step further, the joy-cons could go from being a non-explicit advanced tongue-in-cheek sex toy to being an advanced communications device instead. For instance if a series of precise “rumbles” programmed by the developer could communicate messages, a language could be developed where one could understand what is happening in an application just by feeling the high fidelity rumble feedback messages and taking action, such as through buttons on the controller, voice commands, foot pedals or motion controls. So far this technology has been pioneered by Nintendo, but all of it has been directed towards games and interactive entertainment. It would be interesting to see this technology taken towards business, communications, education and productivity for new forms of accessibility for the blind and a new platform for developers and audiences with potential advantages against other platforms regardless of disability or lack thereof.
Eventually a new technology that doesn’t just give advanced vibration feedback, but is actually able to reconfigure brail, precise tactile messages, images or information that does not need to be seen but touched in order to be understood, one way or another, it would be a new milestone in accessibility and depth in what we can do or learn through these different experiences. This reconfiguration could occur within a limited space on a controller or device that is handled by small motors that re-arrange, based on a program the bumps and grooves to be felt by the finger, instead of displaying an image, giving a tactile alternative to the standard visual interface. Control could be handled either with buttons or keys on the controller or device or by voice-recognized commands and other tactile or motion feedback that is most convenient to the user.
This way not only greater advancements in quiet, alternative and accessible information processing and control can be developed, but the depth at which these technologies can come around full circle and be used back towards the games and interactive entertainment, depending the level of immersion not only in a fantasy world but the amazing world of our own technology today. Combining these with the emerging AR/VR and Mixed reality developments could provide a rich user experience with increased accessibility for people from all walks of life with different situations or conditions.
LAteral IDeas: Chemical 3D Printing OR Modular self-reconfiguring ROBOTS?
We need to continue to find alternatives for greater accessibility to newer technologies that can then be integrated into all-purpose technologies to cultivate more seminal platforms to empower independent growth. When we can master chemical based 3D printing to the point that we could create virtual worlds with physical objects that we could spontaneously create for your current place in an experience, and the same matter then immediately converted into a whole different set of objects for another environment. Though 3D printing of that speed, though chemical processes are advanced, may be very difficult or possibly dangerous, however the concept is taking the advancements of Mixed reality and applying those advancements to the tactile non-seeing world to only expand what can be done. There could be a more practical solution like a self-reassembling modular robot that could turn itself into different tools or interactions for different applications and experiences, aided by AR or Mixed Reality, the possibilities are endless as to what could be done with one such system for example.
The innovation process
As you see the above process is sometimes to think of the crazier ideas or outlandish impractical implementations based on what you know and what you are trying to solve and then work down from there towards more practical and implementable solutions that could change the very way we all live. It goes from impractical and seemingly impossible to help humankind to potentially being among the most helpful, so never be afraid to go there before moving towards resolving your bigger problems. This aspect of going far beyond what is seemingly plausible as what you expect as possible, makes it easier to find creative ideas that may actually work after you reel those ideas back. I hope this article has helped you to think of what innovations will come and how you can be more creative in solving bigger problems and unafraid to move towards working, experimenting and discovering what is most effective in completely new technologies that are still strongly based on what we already know. Good luck! Feel free to contact me on Twitter by mentioning @HonourableHappy in your tweet by sending me your idea.