When our team makes changes to a component in the framework, we want to make sure those changes don’t inadvertently mess up components elsewhere. So, we built a solution for ourselves: Snapshot. Snapshot has been an incredibly valuable tool for tracking all our changes, and we wanted to share how we use it to ensure that Ionic is thoroughly tested to prevent breaking your apps!
What is Snapshot, Anyway?
Snapshot is an internal tool that runs end-to-end tests and takes a screenshot for every command. Each time the tool runs, the Snapshot is assigned a unique ID, which is associated with all of the screenshots taken. Upon completion of the task, the screenshots are uploaded to a server where they can be viewed.
Snapshot uses Protractor to run the commands for each end-to-end test. Protractor is an end-to-end testing tool used by the Angular team to test their apps, and we’re able to use it with Snapshot to visually represent the end-to-end tests. Pretty neat, right?
How Does Ionic Use Snapshot?
Snapshot can be launched in Ionic by running a single Gulp task,
gulp snapshot. The gulp task runs each end-to-end test for three different platforms:
windows. We run Snapshot against all of these platforms, in order to test each mode. Each time we make a change in the framework, we run Snapshot, and check for any visual changes that may have occurred in the framework, in comparison to the Master Snapshot.
What’s the Master Snapshot?
The Master Snapshot is the most recent Snapshot that has been hand-selected and approved by us. Not every Snapshot that gets created will become the Master Snapshot. When we run Snapshot, occasionally we’ll notice a change that we didn’t mean to make. In this case, we’ll fix the problem causing the change, and then run it again. When we review all of the screenshots in a Snapshot and determine that any changes were on purpose, we update the Master Snapshot to use that Snapshot.
The Gulp task uses a Snapshot configuration file that defines the location of the end-to-end tests. In most cases, we’ll want to take screenshots of every end-to-end test in the framework. However, there are times when we’re only concerned with a specific component, and in those cases, we can change the configuration to only run the end-to-end tests for that component. Snapshot can be used to create screenshots from anywhere in our end-to-end tests.
Snapshot can be used to compare any two tests from the interface where our Snapshots are uploaded. To compare two tests, we select the tests we want, using their unique IDs, and then click on “Compare Selected Tests” to see the differences between the tests.
When two tests don’t match, Snapshot will highlight the differences in bright purple. When the change was made on purpose, it’s okay that there is a mismatch between the Master Snapshot and the current Snapshot. In this case, we will update the current Snapshot to be the Master Snapshot, so that when we run Snapshot in the future, it won’t continue to show us a mismatch. The following image shows a change to the framework that was made by fixing a bug. The screenshot on the far left is the Master Snapshot, the middle screen is the screenshot of the latest Snapshot, and the right screenshot shows the differences between the two. After updating the Master Snapshot to the Snapshot containing the screenshot in the middle, all future Snapshots will be compared against that Snapshot.
Sometimes, we make a mistake and change something in the framework that we didn’t mean to change. When this happens, running Snapshot shows us where we messed up, and then we can fix it. The image below shows a change we made to the framework, in which a border was accidentally added to alert. We won’t make this Snapshot the Master Snapshot, since we need to remove the border. After fixing the change, we will run Snapshot again, and once it looks good to us, we will select it as the Master Snapshot.
We built Snapshot to further ensure the framework is continuously well tested, and to reassure you that future versions won’t break your apps. Creating tests for literally all of the scenarios we can think of enables us to catch any pesky bugs in the framework before you or your users encounter them.
We like to think of Snapshot as our safety net before pushing new code live. We do this to make your apps better, and to ensure they’re highly tested before you even start working on them.