Emulators just weren’t cutting it
Like most inventions, the notion of building a ‘device lab’ at Taoti was born out of necessity. As our work evolved, our QA team realized that the device emulators and other tools at their disposal used to test sites from a visual perspective weren’t always accurate. It wasn’t until they realized bugs were getting through the emulators that they came to the conclusion that the only sure-fire way to test visual design for our digital builds was to do it on actual devices. Emulator bugs aside, real devices allowed us to really get a feel for the user experience in a way that an emulator doesn’t. What happens when you swipe in different directions? What’s the transition look like when you switch from portrait to landscape? How do accessibility controls affect the site (aka 508 compliance.) It became clear to us. We needed a device lab if we wanted to take quality control and usability to the next level.
The “Octopus” is born
The big man wanted our best people on this project. But they were busy with client work, so they asked me to spearhead this project. I was tasked with putting together a working team of outcasts and misfits, and we were off to the races. We had three major issues to tackle during the build:
- Create something that would be usable by lots of people (beyond just the QA team)
- Make it mobile (since we need to move this around from time to time)
- Figure out the right mix of equipment to make sure we’re addressing everything we need to without overdoing it.
Ladies and gentlemen, I give to you, the device lab! Or as we like to call it, the Octopus:
Better quality AND lower budgets!
Now back to our initial need – the ability to identify visual deficiencies throughout the build and the final product. The lab solved one massive issue for our team, but another problem it inadvertently addressed was time expenditure. Using the device lab, we’ve created, along with some lightweight software that synchronizes inputs/behavior across all devices simultaneously while testing, increases our QA efficiency by well over 60%. That’s a significant chunk of hours that can’t be ignored – hours that are then redistributed throughout the project aiding to incorporating new enhancements we previously didn’t think we had time for. In conclusion, the time savings alone on multiple projects by utilizing the lab over an emulator has allowed the lab to pay for itself after only its first couple of uses.
Location, location, location….
Politics being politics, my initial recommendation of building another floor on top of our building just for the lab, installing surround sound speakers, a couple of projection screens, and purchasing a vending machine serving only flaming hot Cheetos, paired with a Kool-Aid dispensing water fountain was quickly denied. More practical heads prevailed, and we ended up finding middle ground utilizing a more aptly sized conference room dubbed Dupont Circle (as all conference rooms at Taoti coincide with a metro stop). To handle the viewing of all the devices, we built a new wall in the conference room slanted at a slight grade leaning towards a desk, aiding in visibility to the multitude of devices at eye level. We purchased a mile-long strip of Velcro to stick the devices to the wall securely, while at the same time allowing the ability to rip them off the wall when testing on a device involves a more up close and personal encounter. We mounted a pair of widescreen monitors to the wall (one vertical and one horizontal), and just like that, we had the foundation for the lab.
But this just wasn’t enough. We needed this to be portable for broader team adoption, and more importantly, we needed it to be a tiny bit easier to use. So we got a metal podium, roughly a dozen device holding ‘arms’ (referred to internally as the device octopus), some clever cable management solutions, and created a solution fit for a world-class creative agency like Taoti – take a look!
One device, two devices, three devices, don’t drop the device on the floor
Now that we had the lab shell up in place, we were ready to start figuring out what limited number of devices would be representative of the majority of the users accessing our clients’ sites that we build for them. In a perfect world, we’d have one of every device under the sun, but space being limited, we had to make some hard decisions. What we decided on was to include:
- 3 of the most recent iPhones released
- 1 iPad tablet
- 2 of the most recent Samsung Android phones
- 2 of the most recent Google Pixel phones
- 1 Samsung tablet
- 1 Amazon Kindle tablet
- 14″ and 17″ Laptops running Windows (dual bootable to different versions of Windows OS)
- 13″ and 15″ MacBook Pro
- 2 mounted monitors
- VR headset
- Assorted collection of older devices Apple and Android.
This covers not only the most commonly used devices by our clients’ users, but also tackles one of the most important requirements when testing responsive design, an enormous variety of screen resolutions. To package it all together is the device lab software that synchronizes behavior taken on one device – mobile or desktop – on all of the other devices simultaneously.
Client Feedback
One benefit we frankly didn’t anticipate was the incredible feedback we’ve received from our current (and future at times) clients. Just recently, a client was visiting the Taoti offices, and we decided to throw their site up in the lab so that they can experience their website across all of our devices. Outside of just being super cool, the benefit for us was that it was lending a sense of security and confidence from them in us that we’d leave no stone – nay, device – unturned (pun intended) when testing their site.
Device check out system – Good idea? Or great idea?
Spoiler alert: it’s a great idea. One of the incredible benefits of using such a flexible system to secure our devices to the lab is that it makes it simple for anybody in the office to grab one off the lab and take it to their desk when they’re heads down with their team, trying to drive a bug home. Humans being imperfect (myself excluded, of course) sometimes forget to take the device back at the end of the day. In short, missing device in lab = very uneasy Scott. We threw a device check out system in the lab, and the problem has been minimized, along with my anxiety of one of the devices growing legs and walking out of the lab.
Never stop improving
Our mantra here at Taoti is “wouldn’t it be better if…”, and the Device Lab certainly is no exception to that mindset. Now that we have a mature solution in place, our phase 2 plans for this year include a more complex power management system for the lab, dedicated networking equipment, finding a way to virtualize the lab for remote development staff, increasing device inventory, and last but certainly not least:
Turning our humble Taoti Device into an Open Device Lab, making it one of the only remaining of its kind in the D.C. area.
Stay tuned to our social channels for all future updates regarding the lab, status on procuring the kool-aid fountain and for a small dose of the daily innovation we churn out here at D.C.’s favorite digital agency, Taoti Creative.
