Research existing remote¶
Using an existing bluetooth (low energy) remote saves development time and money, in this research we will analyze an existing remote and document if it works for our use case or why not. The product will be tested on physical properties, behaviour on a smartphone, possibility to be reprogrammed and by using a bluetooth sniffer analyzer to see it’s technical behavior connected to a device.
Physical design¶
The remote after doing some research seems to be designed for usage in social media apps such as “TikTok” or “Instagram”. This results in the physical design having a heart shaped ‘like’ button, voice button (micropone) and Camera buttons. This isn’t exactly optimal if the usage is to be for something completely different. The arrow buttons which are also on the remote are usable for our goal but don’t line up with 1 2 3 4 on the Stephear app but this could be changed obviously.
As the buttons labels are extremely shallow and not braille the fuction of each button are not clear without seeing it, the author at least couldn’t make up what each button meant without seeing, this might be no problem if the user is familiar with function behinds each button but is less than ideal to say the least for new users. Furhter investigation should be done on this.
Reprogramming¶
It was also researched if the device could be reprogrammed but this led no where as the chips on the device are unmarked (the text is scrapped of the ics), besides the EEPROM. Searching the part number of the EEPROM leads to chinese sellers and results in no clear documentation. There is also no way of knowing which value the EEPROM should have as their is no documentation to be found on the bluetooth chip of the remote nor which chip it even is. The USB C port only has power lines to the battery and charging circuit and couldn’t be found on a PC.
Bluetooth commands¶
The device was analyzed with the ‘nRF Connect’ app of Nordic Semiconductors a major firm in the Bluetooth ic industry. This on iPhone led nowhere as the device cannot be found, it then was tried on an Android device which also led nowhere as device again cannot be found in the app despite being connected to the device used for scanning. It is thus likely that the device doesn’t implement bluetooth correctly or isn’t actually a bluetooth le device which seems to be a requirement for the app to find the device.
User experience and behavior¶
Remote behavior connected to a physical device¶
The behaviour thus couldn’t be analyzed with the known programs. Connecting it through a phone shows that the arrow buttons does a swipe and than click action. The other buttons either do nothing (ios) or have platform dependant actions Android and Mac. This behaviour made interacting with the device really strange as it randomly drags and than click if it is on something (but you can’t see what as it it doesn’t show a mouse pointer). This is extremely confusing for normal people let alone blind people as it isn’t clear what is happening nor what is going to happen.
The swipe and click action would be although rather difficult to implement not that big of a problem in the Stephear app itself, the problem comes when the user is for some reason outside of the app. The behaviour than is not favourable, you cannot say to someone that is blind “You can use this remote BUT only in the stephear app and not outside of it”, mistakes are bound to happen especially when used by a (blind) user, when he/she somehow gets out of the app and than has problems with unexpected behaviour because the device doesn’t send logical signals. Swipe actions and then a click are also difficult to catch as multiple actions need to be intercepted after each other and than connected to the wanted action.
Identified action for each button¶
The following table with each buttons corresponding action was identified and the send hid command as far as was possible by using packetlogger.
| Button | Action | HID Command |
|---|---|---|
| Heart | Click | Mouse click 1 & Mouse button 7 with x & y upwards to the left |
| Mic | Nothing (On iOS, Android and macOS at least) | No command registered |
| Camera | Click | Mouse click 1 and button 7 upwards |
| Arrow down | Drag mouse down | Mouse button 7 with x & y downwards than Left click (mouse button 1) |
| Arrow up | Drag mouse up | Mouse button 7 x & y upwards and than Left click (mouse button 1) |
| Arrow left | Drag mouse left | Mouse button 7 x & y to the left and than Left click (mouse button 1) |
| Arrow right | Drag mouse right | Mouse button 7 x & y to the right and than Left click (mouse button 1) |
Conclusion¶
We don’t see the device fit for our use case as it doesn’t have the right physical properties (having descriptive labels) nor is it able to be reprogrammed for better bahaviour on the phone nor can be used in a manner which we can leverage it strengths without having major downsides (expected behavior doesn’t line up with what the user expects). We thus decided to not go further with this idea as we don’t see it worthwhile due to it’s downsides and the time and effort it would take to adapt it to our needs.