Since the early days of the internet, the web development world has been constantly evolving to tackle a question where the answer itself evolves on a seemingly quarterly basis:
“How do I develop a website that behaves exactly as I want it to on every device currently being used by its growing – and often international – userbase?”
First, a quick brief on what emulators/simulators are.
The easiest way to describe them is comparing them to a novelty Atari set that you can buy, pre-loaded with 20-ish classic games that you can plug and play directly into your TV. Now obviously this isn’t the same hardware they used when originally creating the platform in 1972 in lovely Sunnyvale, California. But what this particular piece of hardware does, is use highly efficient and lightweight hardware to simulate the conditions of the old console with some new-age code to back it up, giving the user the same classic experience, without them being able to tell a difference from the vintage user experience they’re accustomed to. However, with a keen eye for detail, you’ll notice the hardware affects the way the software interacts with the user input slightly, even if it’s just in the speed which Mario sprints across the screen being marginally smoother/more different than a younger, chocolate-milk-and-cheese-stick-holding you remembers.
Now, let’s take that logic and apply it to the world of website building.
Let’s assume we’re embarking on an Agile focused project, with iterative QA at development sprint completion. First, the developer completes their code sprint on their local machine, eventually merging it to the master branch after doing some thorough unit testing. A blazing red flare is then shot up into the sky through the projects task/item tracking system, letting the QA team know this feature is ready for QA. At this point, we can assume the QA team has a testing plan and is ready to hit the ground running. Time is of the essence, as you need to tackle every box on your “QA checklist” sheet as efficiently as possible as to not let any remediation tasks roll over to the next sprint. Assuming this web project requires compatibility with the 2 latest Apple phones/tablets, 2 latest Android tablets, 3 mainly used browsers from target audience (let’s assume Chrome, Safari, and IE), you’ve got your work cut out for you. Luckily, you’ve got a simulator/emulator program at your disposal and can tackle the device/browser testing efficiently.
You’re now off to the races!
You sit down at your desk the next morning with a piping hot cup of joe, see the “QA Ready” flare, and you commence the art of meticulously combing through each browser and device through your emulator program, testing each piece of the newly developed (or newly fixed) functionality with a fine-toothed comb. At the end of the day, you’re confident that you’ve now pinpointed every known issue for that piece of functionality, including a thorough walk-through for use cases that your PM has trickled down to you, accompanied by Acceptance Criteria from the client that will need to be met before officially giving the green light. Green light given, and the PM works with the development team to coordinate remediation.
Okay, now, same requirements and task, but it’s the device lab’s time to shine.
You sit down at your desk the next morning, with a piping hot cup of joe, see the “QA Ready” flare, and you commence your QA art – this time, the game is different. Walking into your device lab, you see all of the devices you need to test on (sometimes with multiple versions of that device), charged up and ready to work for you. Fire up the devices to the browsers you wish to test on, whip out your fine-toothed comb and start your process.
The utility of this Device Lab software cannot be overstated.
Not only do we have all the devices we need to test on staring you in the face, but we use some lightweight software to synchronize behavior carried out on any one device, on all devices.
Instead of opening a simulator/emulator one by one that mimics the hardware configuration and software setup of that device, going through your barrage of tests per device/OS/browser and moving on to the next, you get to see all devices and configurations behaving in real time, right in front of you, on real devices – any action taken on the iPhone X as you were scrolling through, is then mimicked on the Galaxy S8, iPad 6, Windows machine (IE11/Edge), etc. You can then take screenshots through the software when an issue is found on any particular device showing this ill behavior, annotate it, download it locally, and add to your task/bug list for your developers review (with a nice detailed set of steps to reproduce, of course). The gravy on this lovely QA feast – also embedded in this screenshot – is the exact device you were using, the OS configuration, screen resolution, date/time of image capture, etc.
Now at the end of the day, all of this makes for an innovative ‘product’ that our sales team can pitch to win us more business. But the true reason we’ve pivoted as an organization to building and utilizing the device lab for the majority of QA-focused activities, is due to the incredible impacts that this has had on our existing client services. We’re able to take the tasks that used to take us a full 8-hour day drowning in the emulators of yesterday and condense that activity into a more efficient – and frankly more accurate – 2 hours of work. I’ll repeat that one more time, this time with gusto:
Using Taoti’s new device lab, we’re able to take 8 hours of work, and turn it into a more accurate 2 hours of work.
We could go into the various implications of saving this amount of time and money on complex builds for websites, but frankly, that statistic speaks for itself. Using this device lab allows us to deliver a new caliber of quality that we regularly promise our clients, while catering to the quality they already expect from us at Taoti.
As vast and complex as the QA process is, that’s certainly not to say that device emulation/simulation isn’t useful during the development process. Services like BrowserStack, Kobitron, etc. will always have their place, especially when one-off testing certain behavior on devices is more efficient than firing up the lab, as firing up a virtual device in an emulator takes 3 clicks and a copy-paste to the address bar.
Setting the device lab up initially cost Taoti a pretty penny. However, because of the time it has saved us and our clients after only its second full-development-cycle use, it has already paid for itself.
More to come detailing our obstacles and logic in how the lab was conceived, a full device list, our plan to open the lab to the public (Opening January 2019), and more in a future QA Chronicle …
